3.6.0

[wangzhaode]

MNN 3.6.0 版本发布说明

发布日期: 2026年6月

📦 安装: pip install mnn==3.6.0 ｜ PyPI

📌 版本概述

MNN 3.6.0 版本聚焦于 多模态 LLM 模型扩展与推理加速、GPU 多后端深度优化 以及 低比特量化与硬件适配增强 三大核心主题：

• 多模态 LLM 模型扩展与推理加速：新增 Gemma4（文本/视觉/语音）、Qwen3.5、Hymt2 等模型支持；推出 dflash 推测解码；Omni 模型 Thinker/Talker 交错生成将 TTFA 降低 2.7×；引入 fp16 存储的 weight-quant scale/zero-point；通过 onClone 重构 KV Cache 复用机制。
• GPU 多后端深度优化：Vulkan 新增 VulkanConv1x1CoopA8（W8A8 coopMat）与 no-subgroup 回退路径；OpenCL 引入 GEMM/GEMV 启发式调度、int4 image1d_buffer_t 优化、原生 TopKV2 与 2/3-bit 权重量化；Metal 优化 Conv1x1 分支选择；CUDA 支持 Blackwell（sm_120）并优化 Eagle 推测解码；QNN 支持 Qwen3.5 与离线图多进程并行导出。
• 低比特量化与硬件适配增强：CPU 与 GPU 全栈支持 2/3-bit 权重量化（ARM low-bit 量化内核 + OpenCL/Metal/Vulkan）；ARMv8.2/v8.6 decode 路径深度优化；KleidiAI 升级到 v1.16.0；新增 RVV 向量化的 Int8/Norm/Gelu/SiLu/ExpC8 等核心算子；新增 fused MNNFusedGatedDelta（LinearAttention prefill）；新增 LinearAttention L=1 decode 快速路径。

---

🚀 版本亮点

• Gemma4 全模态支持：新增 Gemma4 文本、视觉与音频模型的完整推理支持
• Wan2.1-T2V-1.3B 视频生成：新增 Wan2.1-T2V-1.3B 文生视频模型的导出与推理能力
• LLM 推理加速：新增 dflash 推测解码；Omni 模型 Thinker/Talker 交错生成，TTFA 降低 2.7×
• CPU/GPU 2/3-bit 权重量化全栈支持：CPU 新增 low-bit ARM 量化内核，OpenCL/Metal/Vulkan 同步支持 2/3-bit 权重量化
• Vulkan W8A8 coopMat：新增 VulkanConv1x1CoopA8，支持动态量化 W8A8 + Cooperative Matrix

---

✨ 新功能

LLM / VLM

• Gemma4 模型支持：新增 Gemma4 文本、视觉、音频全模态推理
• dflash 推测解码：新增 dflash 推测解码方案
• Omni Thinker/Talker 交错生成：Omni 模型 Thinker 与 Talker 交错执行，TTFA 降低 2.7×
• fp16 Scale/Zero-Point 存储：权重量化的 scale/zero-point 支持 fp16 存储
• KV Cache onClone 重构：通过 onClone 统一各后端 KV Cache 复用逻辑
• LLM log 保存到 context：支持将 LLM 推理日志保存到 context
• linear_attention 视为 full attention：性能优化路径

模型导出 / 模型支持

• Gemma4 模型导出：支持 Gemma4 多模态导出
• Stable Diffusion 3.5 Medium：新增 SD 3.5 Medium 模型支持
• Wan2.1-T2V-1.3B 视频模型：新增视频生成模型的导出与推理

GPU 后端

• Vulkan VulkanConv1x1CoopA8：新增 W8A8 动态量化 + Cooperative Matrix 卷积
• Vulkan LinearAttention 无 subgroup 回退：兼容不支持 subgroup 的设备
• Vulkan gemv decode 无 subgroup 回退：decode 路径的兼容性增强
• Vulkan 持久状态跨响应复用：支持持久状态在多次响应间复用
• Vulkan LayerNorm onClone：支持 LayerNorm onClone，opt pipeline 构造时初始化
• OpenCL GEMM/GEMV 启发式调度：量化 GEMM/GEMV 启发式 + XGEMM FP local size 设置
• OpenCL int4 GEMM/GEMV 优化：使用 image1d_buffer_t 优化 int4 GEMM/GEMV，免除 dequant 时额外的 -8
• OpenCL 原生 TopKV2：OpenCL IMAGE 与 Vulkan（image/buffer）新增原生 TopKV2 实现
• OpenCL LinearAttention chunk 分支优化：优化 chunk 分支性能
• OpenCL/Metal/Vulkan 2/3-bit 权重量化：GPU 三后端统一支持 2/3-bit 权重量化（CPU 端通过 low-bit ARM 量化内核支持）
• Metal Conv1x1 分支选择优化：MetalConvolution1x1 分支路径优化
• CUDA Blackwell (sm_120) 支持：新增 sm_120 计算架构支持
• CUDA Eagle 推测解码优化：CUDA Eagle 推测解码性能优化
• QNN Qwen3.5 支持：QNN 后端支持 Qwen3.5 及视觉模型修复
• QNN 离线图多进程并行导出：使用多进程并行加速离线图导出
• QNN Binary 算子优化：加速视觉模型的 binary op 路径

CPU 后端

• RVV Int8 加速：新增 RVV 加速的 Int8 函数
• RVV 通用 opt 函数：新增 RVV 向量化的 MNNNorm、MNNGelu、MNNSiLu、MNNExpC8 等
• ARM v8.2/v8.6 Decode 优化：手机端 LLM decode 性能显著提升
• LinearAttention prefill fused kernel：新增 MNNFusedGatedDelta fused kernel
• LinearAttention L=1 decode 快速路径：新增 decode 快速路径
• Low-bit ARM 量化内核：CPU 新增 2/3-bit 等低比特 ARM 量化内核支持
• fp16 alpha lowMemory 优化：跳过 fp32 round-trip
• int4 reorder 性能优化：减少非 fast path 的 int4 reorder 开销

构建 / 测试 / 工具

• MNN 发布工作流升级：新增 LLM/OMNI、Harmony、Android 16K 页大小支持
• test_ci.sh：新增统一 CI 入口脚本
• CMake 生成 podspec：从 CMake 自动生成 podspec
• GEMM 性能基准：新增 FP32/INT8/Block32-INT4 GEMM 基准
• GEMV 带宽基准：新增 CPU/Metal w8/w4/w3/w2 GEMV 带宽基准

应用

• MNNChat Android 0.8.3 模型市场：为 Android 用户准备 0.8.3 版本模型目录

---

🐛 缺陷修复

LLM / VLM

• Qwen3.5 head_dim 自动检测：修复 gated attention 模型的 head_dim 自动检测
• LinearAttention prefix disk cache：修复 LinearAttention prefix disk cache 精度及混合模型支持问题
• prefix kvcache 损坏崩溃：修复 prefix kvcache 损坏导致的崩溃以及 expandKVCacheInDisk 中的 size_t 截断
• 推测解码 cache mismatch：修复推测解码中的 cache mismatch 与 mrope 支持
• vision encoder 内存泄露：修复 vision encoder 内存泄露与延迟增长问题
• llm_demo linear-attention 状态重置：在 llm_demo 中重置 linear-attention 状态
• kvmeta 内存泄露：修复 kvmeta 内存泄露
• stop_ids fallback：修复 list 形式 eos_token_id 的 stop_ids fallback
• GPTQ weight_dict 重初始化：修复 GPTQ loader 中的 weight_dict 重初始化问题
• GTE embedding/reranker RoPE：恢复 GTE embedding/reranker 导出中清零的 RoPE inv_freq buffer
• Gemma3 dual-RoPE 导出：修复 Gemma3 dual-RoPE 导出与 Qwen3-VL bilinear 采样
• Gemma4 导出：修复 HF transformers>=5.5.4 纯 KV sharing 导致的 gemma4 导出崩溃；通过 mapper 隔离 k_eq_v 与 head_dim 自动检测
• Hymt2 qk norm 导出：修复 Hymt2 qk norm 导出处理
• Qwen2.5-Omni 音频预处理：修复 Qwen2.5-Omni 音频预处理的边界情况
• Qwen3-VL DeepStack dtype mismatch：修复 Qwen3-VL DeepStack dtype 不一致问题
• Qwen3-VL 重复输出：通过 per-RTM KVCACHE_INFO meta 修复 Qwen3-VL 重复输出问题
• _maskQK gapLen：将 gapLen 计算移动到 null mask early return 之后

GPU 后端

• Vulkan attention 及其他算子修复
• Vulkan 静态符号查找改为动态符号
• Metal output reuse offset 错误：修复 Metal 输出复用 offset 错误
• Metal API 兼容性保护：新增 Metal API 兼容性保护
• OpenCL Attention 越界访问：修复 Attention 算子在内存重分配前的越界访问
• OpenCL LinearAttention GQA bug：修复 OpenCL LinearAttention 的 GQA bug
• CUDA Softmax echo / FP16FP32Mix：修复 FP16FP32Mix 模式下 Softmax 回声伪影与 MatMul precisionType 问题
• CUDA Windows / MNN_CUDA_NATIVE_ARCH 编译：修复 CUDA 在 Windows 与 MNN_CUDA_NATIVE_ARCH 下的 CMake 构建失败
• QNN MSVC 编译：修复 QNN 后端及 MNN2QNNModel 在 Windows MSVC 下的编译错误

CPU 后端

• VNNI w4 < 64 计算错误：修复 VNNI w4 < 64 的计算错误
• KleidiAI int4 dequant：升级 KleidiAI 到 v1.16.0，修复 block-quantized int4 模型的反量化错误
• KleidiAI 路径映射：修复将 MNN 作为子项目时的 KleidiAI 路径映射
• KleidiAI depthwise SME2 guard：为 KleidiAI depthwise conv 路径添加 SME2 guard
• RVV pack/unpack 错误：修复 RVV pack/unpack 函数错误
• CPU sdot fallback：避免在 Memory_Low 外强制 fp32 fallback（CPU 无 sdot 时）
• cstdint 包含缺失：补全 cstdint 头文件
• cpufreq 扫描错误日志：修复扫描 cpufreq policy 目录时的错误日志

算子 / Converter

• Unique 算子：修复 Unique 算子使输出符合 ONNX 排序规范；修复 index 输出问题
• TFLite binary activation：修复 TFLite 转换器中 binary 激活的转换
• ONNX Resize / Einsum：修复 ONNX Resize 转换器回归与 Einsum 4D 非对齐 batch 维快速路径
• cumsum int 类型不一致：修复 cumsum int 类型不一致与 virtual type command 错误

Diffusion

• SD1.5 resize cache warmup：修复 SD1.5 resize cache warmup
• tokenizer.hpp cstdint：修复 tokenizer.hpp 中 uint8_t 未声明错误
• diffusion llm link sep-build：修复 sep-build 模式下的 llm 链接问题

编译 / Embedding / 其它

• embedding_demo 崩溃：修复 mMeta->add==0 导致的 embedding_demo 崩溃
• PyMNN llm 子模块：使 PyMNN 中的 llm 子模块可选导入
• Windows PyMNN .pdb 膨胀：移除 Windows 上的 /Zi 和 /DEBUG 标志，避免 .pdb 膨胀 wheel
• dflash Windows MSVC 编译：修复 dflash 在 Windows MSVC 下的编译错误
• 通用编译问题修复：修复 commit f40b3d2a7 引入的若干编译问题

---

📚 其他改进

• AGENTS.md：将 CLAUDE.md 重命名为 AGENTS.md 并通过 include 转发
• VirtualFaceCapture-MNN 加入 demo 列表
• MNN_CUDA_NATIVE_ARCH 文档：补充 MNN_CUDA_NATIVE_ARCH cmake 选项的文档
• fp16 scale schema 同步
• 代码清理：删除 MNN_CPU_DEQUANT_GEMM：移除已废弃的 MNN_CPU_DEQUANT_GEMM 相关代码
• 代码清理：删除 WEIGHT_DEQUANT_GEMM 宏：移除废弃宏，精简代码

---

📊 性能基准

测试环境: Mac mini M4 (16GB) / OnePlus PJZ110 (Snapdragon 8 Elite) | 模型: Qwen3-0.6B / Qwen3-4B | 量化: W4 (block=64)

🙏 致谢

衷心感谢所有贡献者对本版本的宝贵贡献（按 GitHub login 列出）：

• @jxt1234 — ARMv8.2/v8.6 decode 优化 ([CPU:Speed] Opt armv8.2 and armv8.6 for decode speed, for phone speed up #4389)、Metal Conv1x1 分支选择优化 ([Metal:Speed] Optimize the branch select for MetalConvolution1x1, #4520)、Metal output reuse offset 修复 (修正使用 Metal 后端时，module 在 output 被占用情况下第二次推理后，首次推理的 output offset 被异常覆盖，导致后续结果错误的问题 #4365)、VNNI w4 < 64 修复 (MNN:Bugfix: Fix bug for VNNI w4 < 64 compute error #4366)、int4 reorder 优化 ([CPU:Speed] Reduce not fast path for int4 reorder #4511)、Geometry cumsum virtual type 修复 ([Geometry:Bugfix] Fix bug for cumsum use command for virutal type #4391)
• @Fatemanx (xly) — Wan2.1-T2V-1.3B 视频模型 ([Diffusion:Feature] Wan2.1-T2V-1.3B video model export and inference support #4469)、CUDA Blackwell sm_120 ([CUDA:Feature] Support Blackwell (sm_120) compute architecture #4467)、Qwen2.5-Omni 音频边界修复 (Fix Qwen2.5-Omni audio preprocessing edge cases #4459)、Qwen3-VL DeepStack dtype 修复 ([LLM:Bugfix] Fix Qwen3-VL Python DeepStack dtype mismatch #4504)、Hymt2 qk norm 导出 ([LLM] Support Hy-MT2-1.8B export and inference #4490)、SD1.5 resize cache warmup ([Diffusion:Bugfix] Fix SD1.5 resize cache warmup #4530)、PyMNN llm 可选导入 ([PyMNN:Bugfix] Make llm submodule import optional #4468)、Converter Einsum 4D fast-path ([Converter:Bugfix] Fix Einsum 4D fast-path for non-aligned batch dims #4466)
• @EricMoin — Omni Thinker/Talker 交错生成，TTFA ↓2.7× ([LLM:Perf] Interleaved Thinker/Talker generation for Omni — TTFA ↓2.7x #4397)
• @luckysmg — LinearAttention L=1 decode fast path ([LLM.CPULinearAttention]Decode在L=1下的快速路径 #4356)、MNNFusedGatedDelta prefill fused kernel ([CPU:Perf] add fused MNNFusedGatedDelta kernel for LinearAttention prefill #4434)、LinearAttention prefix disk cache 精度修复 ([LLM:Bugfix] Fix CPULinearAttention prefix disk cache accuracy and hybrid model support #4495)、prefix kvcache 崩溃 / expandKVCacheInDisk size_t 修复 ([LLM,CPU:Bugfix] Fix crash on corrupted prefix kvcache and size_t narrowing in expandKVCacheInDisk #4413)
• @hebangwen — vision encoder 内存泄露与延迟增长修复 ([LLM:Bugfix] fix vision encoder memory leaks and latency growth #4441)、llm_demo linear-attention 状态重置 ([LLM:Bugfix] Reset linear-attention states in llm_demo #4424)
• @LudovicoYIN (YinHanke) — ONNX Resize / Einsum lowering 修复 ([MNNConvert:Bugfix] Fix ONNX Resize optional inputs and Einsum weighted sum lowering #4254)
• @cgisky1980 (顾真牛) — CUDA Softmax echo / FP16FP32Mix 修复 (Fix CUDA Softmax echo artifacts and MatMul precisionType bug #4476)、CUDA Windows / MNN_CUDA_NATIVE_ARCH 构建修复 (Fix CUDA backend CMake build on Windows #4477)
• @yydhYYDH (YYDH) — GPTQ loader weight_dict 重初始化修复 ([LLM:Bugfix] Fix weight_dict reinitialization in GPTQ loader #4411)、QNN 离线图多进程并行导出 ([QNN:Perf] Parallelize offline graph export with multi-process #4414)
• @HenryDen — KleidiAI v1.16.0 升级修复 int4 dequant ([CPU:Bugfix] Upgrade KleidiAI to v1.16.0 to fix incorrect dequantizat… #4474)、KleidiAI 子项目路径映射修复 ([CPU:Bugfix]Fix the KleidiAI path mapping while using MNN as a subpro… #4542)
• @luke-opus (liushengtong) — RVV 向量化 MNNNorm/MNNGelu ([CPU:Feature] RVV: Add vectorized MNNNorm and MNNGelu implementations #4508)、RVV 向量化 MNNSiLu/MNNExpC8 ([CPU:Feature] RVV: Add vectorized MNNSiLu and MNNExpC8 implementations #4484)
• @Sherlockzhangjinge — RVV 加速 Int8 函数 (Feature/rvv support #4359)
• @typer-J (XJJ) — RVV 通用 opt 函数优化 (RVV: Optimize common opt functions #4433)
• @ihb2032 — RVV pack/unpack 函数错误修复 ([CPU:Bugfix] fix rvv pack and unpack functions errors #4531)
• @Juude (Jinde.Song) — MNNChat Android 0.8.3 模型市场准备 (Add the 0.8.3 model-market entries for Gemma 4, LFM/LFM2.5, and Qwen3.5 Claude 4.6 Opus reasoning-distilled models. #4405)
• @devin-lai (Devin Lai) — Unique index 输出修复 ([CPU:Bugfix] Fix Unique index outputs #4529)、TFLite binary activation 修复 ([Converter:Bugfix] Fix TFLite binary activation conversion #4540)
• @EduMenges (Eduardo Menges Mattje) — Vulkan 静态符号查找改为动态 ([Vulkan:Bugfix] Removed static lookup to dynamic symbol #4348)
• @DavidVentura (David Ventura) — CPU sdot fallback 修复 (Don't force fp32 fallback outside Memory_Low on CPUs without sdot #4534)
• @katolikov (Artyom Katolikov) — test_ci.sh 与 Vulkan/OpenCL 算子修复 (Add test_ci.sh + declarative stage config (test_stages.json) and assorted CI / op fixes #4422)
• @zlaazlaa (zlaa) — Stable Diffusion 3.5 Medium 支持 ([Model:Feature] Add Stable Diffusion 3.5 Medium support #4072)
• @Voine — VirtualFaceCapture-MNN 加入 demo 列表 (Add VirtualFaceCapture-MNN link to demo.md #4497)
• @Dugtx — Diffusion sep-build 模式 llm 链接修复 ([Diffusion:Bugfix] Fix LLM link in sep-build mode #4438)
• @Saisimon — cstdint 头文件修复 ([CPU:Bugfix]Add cstdint include for fixed-width integer types #4338)
• @Dts0 (dream zhao) — tokenizer.hpp cstdint 修复 ([Vulkan:Bugfix] Fix bug for building mnncli with -DMNN_VULKAN=ON #4373)
• @tumuyan — cpufreq policy 错误日志修复 ([CPU:Chore] Fix spurious error log when scanning cpufreq policy direc… #4400)

---

MNN 3.6.0 Release Notes

Release Date: June 2026

📦 Install: pip install mnn==3.6.0 ｜ PyPI

📌 Overview

MNN 3.6.0 focuses on three core themes: Multimodal LLM Model Expansion & Inference Acceleration, Deep Multi-Backend GPU Optimization, and Low-bit Quantization & Hardware Adaptation Enhancement:

• Multimodal LLM Model Expansion & Inference Acceleration: Added support for Gemma4 (text/vision/audio), Qwen3.5, Qwen3-VL DeepStack, and Hymt2 models; introduced dflash speculative decoding; Omni Thinker/Talker interleaved generation reduces TTFA by 2.7×; new fp16 storage for weight-quant scale/zero-point; refactored KV Cache reuse via onClone.
• Deep Multi-Backend GPU Optimization: Vulkan adds VulkanConv1x1CoopA8 (W8A8 coopMat) and no-subgroup fallback paths; OpenCL adds heuristic GEMM/GEMV scheduling, int4 image1d_buffer_t optimization, native TopKV2 and 2/3-bit weight quantization; Metal optimizes Conv1x1 branch selection; CUDA supports Blackwell (sm_120) and optimizes Eagle speculative decoding; QNN supports Qwen3.5 and multi-process parallel offline graph export.
• Low-bit Quantization & Hardware Adaptation Enhancement: Full-stack 2/3-bit weight quantization across CPU and GPU (ARM low-bit quant kernels + OpenCL/Metal/Vulkan); deep optimization of ARMv8.2/v8.6 decode path; KleidiAI upgraded to v1.16.0; new RVV-vectorized Int8/Norm/Gelu/SiLu/ExpC8 kernels; new fused MNNFusedGatedDelta (LinearAttention prefill); new LinearAttention L=1 decode fast path.

---

🚀 Highlights

• Gemma4 Multimodal Support: Full inference support for Gemma4 text, vision, and audio
• Wan2.1-T2V-1.3B Video Generation: Export and inference support for Wan2.1-T2V-1.3B text-to-video model
• LLM Inference Acceleration: New dflash speculative decoding; Omni Thinker/Talker interleaved generation reduces TTFA by 2.7×
• Full-Stack 2/3-bit Weight Quantization: 2/3-bit weight quantization across CPU (low-bit ARM kernels) and GPU (OpenCL/Metal/Vulkan)
• Vulkan W8A8 coopMat: New VulkanConv1x1CoopA8 with dynamic-quant W8A8 + Cooperative Matrix

---

✨ New Features

LLM / VLM

• Gemma4 Model Support: Full text + vision + audio multimodal inference
• dflash Speculative Decoding: New dflash speculative decoding
• Omni Interleaved Thinker/Talker: TTFA reduced by 2.7×
• fp16 Scale/Zero-Point Storage: weight-quant scale/zero-point can be stored in fp16
• KV Cache onClone Refactor: Unified KV Cache reuse via onClone across backends
• LLM log save to context: Persist LLM inference logs to context
• Treat linear_attention as full attention: Performance optimization path

Model Export / Model Support

• Gemma4 Export: Multimodal export support
• Stable Diffusion 3.5 Medium
• Wan2.1-T2V-1.3B: Video generation model export and inference

GPU Backends

• Vulkan VulkanConv1x1CoopA8: Dynamic-quant W8A8 + Cooperative Matrix conv
• Vulkan LinearAttention no-subgroup fallback
• Vulkan gemv decode no-subgroup fallback
• Vulkan persistent states across responses
• Vulkan LayerNorm onClone: Move opt pipeline creation into constructor
• OpenCL Heuristic GEMM/GEMV Scheduling: Heuristic guidance for quantized GEMM/GEMV + XGEMM FP local size
• OpenCL int4 GEMM/GEMV Optimization: image1d_buffer_t-based int4 GEMM/GEMV, removing the extra -8 during dequant
• OpenCL Native TopKV2: For OpenCL IMAGE and Vulkan (image/buffer)
• OpenCL LinearAttention Chunk Branch Optimization
• OpenCL/Metal/Vulkan 2/3-bit Weight Quantization: Unified across GPU backends (CPU supported via low-bit ARM quant kernels)
• Metal Conv1x1 Branch Selection Optimization
• CUDA Blackwell (sm_120) Support
• CUDA Eagle Speculative Decoding Optimization
• QNN Qwen3.5 Support with abnormal quant params and vision model fixes
• QNN Multi-Process Parallel Offline Graph Export
• QNN Binary Op Optimization: Speeds up vision models

CPU Backend

• RVV Int8 Acceleration
• RVV General Opt Functions: MNNNorm, MNNGelu, MNNSiLu, MNNExpC8
• ARM v8.2/v8.6 Decode Optimization
• LinearAttention prefill fused kernel: MNNFusedGatedDelta
• LinearAttention L=1 Decode Fast Path
• Low-bit ARM Quant Kernels: CPU 2/3-bit and other low-bit ARM quant kernel support
• fp16 alpha lowMemory Optimization: Skip fp32 round-trip
• int4 Reorder Optimization: Reduce non-fast-path overhead

Build / Testing / Tools

• Release Workflow Upgrade: LLM/OMNI, Harmony, Android 16K page size
• test_ci.sh: Unified CI entry script
• CMake-generated podspec
• GEMM Benchmarks: FP32 / INT8 / Block32-INT4
• GEMV Bandwidth Benchmarks: CPU/Metal w8/w4/w3/w2

Apps

• MNNChat Android 0.8.3 Model Catalog Preparation

---

🐛 Bug Fixes

LLM / VLM

• Qwen3.5 head_dim Auto-Detection: Fixed head_dim auto-detection for gated attention models
• LinearAttention prefix disk cache: Fixed accuracy and hybrid model support
• Crash on corrupted prefix kvcache: Fixed crash on corrupted prefix kvcache and size_t narrowing in expandKVCacheInDisk
• Speculative decoding cache mismatch: Fixed cache mismatch and mrope support
• Vision encoder memory leak and latency growth: Fixed
• llm_demo linear-attention state reset: Reset linear-attention states in llm_demo
• kvmeta memory leak: Fixed
• stop_ids fallback: Fixed list-form eos_token_id stop_ids fallback
• GPTQ weight_dict reinitialization: Fixed
• GTE embedding/reranker RoPE: Restored zeroed RoPE inv_freq buffer
• Gemma3 dual-RoPE export & Qwen3-VL bilinear sampling: Fixed
• Gemma4 export: Fixed crash with HF transformers>=5.5.4 pure KV sharing; isolated k_eq_v and head_dim auto-detection via mapper
• Hymt2 qk norm export: Fixed
• Qwen2.5-Omni audio preprocessing: Fixed edge cases
• Qwen3-VL DeepStack dtype mismatch: Fixed
• Qwen3-VL repeating output: Fixed via per-RTM KVCACHE_INFO meta
• _maskQK gapLen: Moved gapLen computation after null mask early return

GPU Backends

• Vulkan attention and other op fixes
• Vulkan dynamic symbol lookup
• Metal output reuse offset error: Fixed
• Metal API compatibility protection: Added
• OpenCL Attention OOB access: Fixed OOB access before memory reallocation
• OpenCL LinearAttention GQA bug: Fixed
• CUDA Softmax echo / FP16FP32Mix: Fixed echo artifacts and MatMul precisionType in FP16FP32Mix mode
• CUDA Windows / MNN_CUDA_NATIVE_ARCH Build: Fixed CMake build failures
• QNN MSVC Build: Fixed QNN backend and MNN2QNNModel MSVC compile errors

CPU Backend

• VNNI w4 < 64 Compute Error: Fixed
• KleidiAI int4 dequant: Upgraded to v1.16.0 to fix incorrect block-quantized int4 dequant
• KleidiAI Path Mapping: Fixed when using MNN as a subproject
• KleidiAI depthwise SME2 guard: Added
• RVV pack/unpack Errors: Fixed
• CPU sdot Fallback: Don't force fp32 fallback outside Memory_Low on CPUs without sdot
• Missing cstdint Include: Added
• cpufreq Scan Error Log: Fixed spurious error log when scanning cpufreq policy directories

Ops / Converter

• Unique Op: Fixed to produce sorted output per ONNX spec; fixed index outputs
• TFLite binary activation conversion: Fixed
• ONNX Resize / Einsum: Fixed ONNX Resize converter regressions and Einsum 4D fast-path for non-aligned batch dims
• cumsum int Type Mismatch: Fixed cumsum int type mismatch and virtual type command bug

Diffusion

• SD1.5 resize cache warmup: Fixed
• tokenizer.hpp cstdint: Added include to fix uint8_t undeclared
• Diffusion llm link in sep-build mode: Fixed

Build / Embedding / Misc

• embedding_demo crash: Fixed crash caused by mMeta->add==0
• PyMNN llm submodule: Made llm submodule import optional
• Windows PyMNN .pdb bloat: Removed /Zi and /DEBUG flags on Windows
• dflash Windows MSVC Build: Fixed dflash Windows MSVC compile error
• General compilation fixes: Fixed compilation issues introduced by commit f40b3d2a7

---

📚 Other Improvements

• AGENTS.md: Renamed CLAUDE.md to AGENTS.md and forwarded via include
• VirtualFaceCapture-MNN added to demo list
• MNN_CUDA_NATIVE_ARCH Documentation
• fp16 scale schema sync
• Code cleanup: removed MNN_CPU_DEQUANT_GEMM: Deleted deprecated MNN_CPU_DEQUANT_GEMM code
• Code cleanup: removed WEIGHT_DEQUANT_GEMM macro: Removed deprecated macro

---

🙏 Acknowledgements

We sincerely thank all contributors for their valuable contributions to this release (listed by GitHub login):

• @jxt1234 — ARMv8.2/v8.6 decode optimization ([CPU:Speed] Opt armv8.2 and armv8.6 for decode speed, for phone speed up #4389), Metal Conv1x1 branch selection ([Metal:Speed] Optimize the branch select for MetalConvolution1x1, #4520), Metal output reuse offset fix (修正使用 Metal 后端时，module 在 output 被占用情况下第二次推理后，首次推理的 output offset 被异常覆盖，导致后续结果错误的问题 #4365), VNNI w4 < 64 fix (MNN:Bugfix: Fix bug for VNNI w4 < 64 compute error #4366), int4 reorder optimization ([CPU:Speed] Reduce not fast path for int4 reorder #4511), Geometry cumsum virtual type fix ([Geometry:Bugfix] Fix bug for cumsum use command for virutal type #4391)
• @Fatemanx (xly) — Wan2.1-T2V-1.3B video model ([Diffusion:Feature] Wan2.1-T2V-1.3B video model export and inference support #4469), CUDA Blackwell sm_120 ([CUDA:Feature] Support Blackwell (sm_120) compute architecture #4467), Qwen2.5-Omni audio edge cases (Fix Qwen2.5-Omni audio preprocessing edge cases #4459), Qwen3-VL DeepStack dtype fix ([LLM:Bugfix] Fix Qwen3-VL Python DeepStack dtype mismatch #4504), Hymt2 qk norm export ([LLM] Support Hy-MT2-1.8B export and inference #4490), SD1.5 resize cache warmup ([Diffusion:Bugfix] Fix SD1.5 resize cache warmup #4530), PyMNN llm optional import ([PyMNN:Bugfix] Make llm submodule import optional #4468), Converter Einsum 4D fast-path ([Converter:Bugfix] Fix Einsum 4D fast-path for non-aligned batch dims #4466)
• @EricMoin — Omni Thinker/Talker interleaved generation, TTFA ↓2.7× ([LLM:Perf] Interleaved Thinker/Talker generation for Omni — TTFA ↓2.7x #4397)
• @luckysmg — LinearAttention L=1 decode fast path ([LLM.CPULinearAttention]Decode在L=1下的快速路径 #4356), MNNFusedGatedDelta prefill fused kernel ([CPU:Perf] add fused MNNFusedGatedDelta kernel for LinearAttention prefill #4434), LinearAttention prefix disk cache accuracy fix ([LLM:Bugfix] Fix CPULinearAttention prefix disk cache accuracy and hybrid model support #4495), prefix kvcache crash / expandKVCacheInDisk size_t fix ([LLM,CPU:Bugfix] Fix crash on corrupted prefix kvcache and size_t narrowing in expandKVCacheInDisk #4413)
• @hebangwen — Vision encoder memory leak / latency growth fix ([LLM:Bugfix] fix vision encoder memory leaks and latency growth #4441), llm_demo linear-attention state reset ([LLM:Bugfix] Reset linear-attention states in llm_demo #4424)
• @LudovicoYIN (YinHanke) — ONNX Resize / Einsum lowering fix ([MNNConvert:Bugfix] Fix ONNX Resize optional inputs and Einsum weighted sum lowering #4254)
• @cgisky1980 (顾真牛) — CUDA Softmax echo / FP16FP32Mix fix (Fix CUDA Softmax echo artifacts and MatMul precisionType bug #4476), CUDA Windows / MNN_CUDA_NATIVE_ARCH build fix (Fix CUDA backend CMake build on Windows #4477)
• @yydhYYDH (YYDH) — GPTQ loader weight_dict reinit fix ([LLM:Bugfix] Fix weight_dict reinitialization in GPTQ loader #4411), QNN multi-process parallel offline graph export ([QNN:Perf] Parallelize offline graph export with multi-process #4414)
• @HenryDen — KleidiAI v1.16.0 upgrade fixing int4 dequant ([CPU:Bugfix] Upgrade KleidiAI to v1.16.0 to fix incorrect dequantizat… #4474), KleidiAI subproject path mapping fix ([CPU:Bugfix]Fix the KleidiAI path mapping while using MNN as a subpro… #4542)
• @luke-opus (liushengtong) — RVV vectorized MNNNorm/MNNGelu ([CPU:Feature] RVV: Add vectorized MNNNorm and MNNGelu implementations #4508), RVV vectorized MNNSiLu/MNNExpC8 ([CPU:Feature] RVV: Add vectorized MNNSiLu and MNNExpC8 implementations #4484)
• @Sherlockzhangjinge — RVV accelerated Int8 functions (Feature/rvv support #4359)
• @typer-J (XJJ) — RVV common opt functions optimization (RVV: Optimize common opt functions #4433)
• @ihb2032 — RVV pack/unpack functions fix ([CPU:Bugfix] fix rvv pack and unpack functions errors #4531)
• @Juude (Jinde.Song) — MNNChat Android 0.8.3 model catalog preparation (Add the 0.8.3 model-market entries for Gemma 4, LFM/LFM2.5, and Qwen3.5 Claude 4.6 Opus reasoning-distilled models. #4405)
• @devin-lai (Devin Lai) — Unique index outputs fix ([CPU:Bugfix] Fix Unique index outputs #4529), TFLite binary activation fix ([Converter:Bugfix] Fix TFLite binary activation conversion #4540)
• @EduMenges (Eduardo Menges Mattje) — Vulkan static → dynamic symbol lookup ([Vulkan:Bugfix] Removed static lookup to dynamic symbol #4348)
• @DavidVentura (David Ventura) — CPU sdot fallback fix (Don't force fp32 fallback outside Memory_Low on CPUs without sdot #4534)
• @katolikov (Artyom Katolikov) — test_ci.sh and Vulkan/OpenCL op fixes (Add test_ci.sh + declarative stage config (test_stages.json) and assorted CI / op fixes #4422)
• @zlaazlaa (zlaa) — Stable Diffusion 3.5 Medium support ([Model:Feature] Add Stable Diffusion 3.5 Medium support #4072)
• @Voine — Added VirtualFaceCapture-MNN to demo list (Add VirtualFaceCapture-MNN link to demo.md #4497)
• @Dugtx — Diffusion sep-build mode llm link fix ([Diffusion:Bugfix] Fix LLM link in sep-build mode #4438)
• @Saisimon — cstdint header fix ([CPU:Bugfix]Add cstdint include for fixed-width integer types #4338)
• @Dts0 (dream zhao) — tokenizer.hpp cstdint fix ([Vulkan:Bugfix] Fix bug for building mnncli with -DMNN_VULKAN=ON #4373)
• @tumuyan — cpufreq policy error log fix ([CPU:Chore] Fix spurious error log when scanning cpufreq policy direc… #4400)

---

Full Changelog: 3.5.0...3.6.0

---

This discussion was created from the release 3.6.0.

[hhoking]

能否告知windows的release包是用vs哪个版本编译的？vs2019或更高版本？另，以后发布release包能否将编译环境说明也附带一下？多谢。

[wangzhaode]

MNN 3.6.0 的 Windows release 包是通过 GitHub Actions windows-release job 编译的。该 job 使用 windows-latest runner，当次实际 runner 镜像为 windows-2025-vs2026，对应 Visual Studio 2026 Enterprise，MSVC toolset 14.51.36231，CMake 识别的编译器版本为 MSVC 19.51.36246.0。后续 release 包我们可以补充编译环境说明，避免 windows-latest 这类浮动镜像带来的歧义。

[hhoking]

非常感谢您的回复，主要看到接口里有用到C++标准库的类型，觉得应该得对齐一下编译版本，避免一些不必要的麻烦。再次感谢！ ---- Replied Message ---- ***@***.***>Date6/30/2026 ***@***.***>***@***.***>, ***@***.***>SubjectRe: [alibaba/MNN] 3.6.0 (Discussion #4546) MNN 3.6.0 的 Windows release 包是通过 GitHub Actions windows-release job 编译的。该 job 使用 windows-latest runner，当次实际 runner 镜像为 windows-2025-vs2026，对应 Visual Studio 2026 Enterprise，MSVC toolset 14.51.36231，CMake 识别的编译器版本为 MSVC 19.51.36246.0。后续 release 包我们可以补充编译环境说明，避免 windows-latest 这类浮动镜像带来的歧义。 — Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications, keep track of coding agent tasks and review pull requests on the go with GitHub Mobile for iOS and Android. Download it today! You are receiving this because you commented.Message ID: ***@***.***>