Skip to content

[Infra:Chore] Sync Alibaba MNN 3.6.1 - #4

Merged
pruthvikar merged 37 commits into
cleanup/remove-unused-apps-and-projectsfrom
feature/sync-upstream-mnn
Aug 6, 2026
Merged

[Infra:Chore] Sync Alibaba MNN 3.6.1#4
pruthvikar merged 37 commits into
cleanup/remove-unused-apps-and-projectsfrom
feature/sync-upstream-mnn

Conversation

@pruthvikar

@pruthvikar pruthvikar commented Aug 4, 2026 •
edited
Loading

Copy link
Copy Markdown
Member

Description

  • Merge Alibaba MNN master through 256a3ae2, bringing in 35 upstream commits since the fork's previous upstream merge at df6f6490.
  • Update to MNN 3.6.1 and import upstream Apple Metal LLM optimizations, OpenCL/CUDA/QNN/Hexagon fixes and features, LLM exporter/runtime improvements, and operator memory-safety fixes.
  • Preserve the Carv cleanup branch: the deleted Android demo path stays removed, while the PyMNN package configuration keeps the compact local formatting and adopts upstream's numpy<2.5 and manylinux2014 changes.
  • Repair CI paths invalidated by the cleanup: Android and iOS now build through CMake, wiki publishing is limited to alibaba/MNN, and PyMNN reports the expected TypeError after raw integer-pointer support was removed upstream.
  • Keep the KleidiAI fix independent: upstream did not change cmake/KleidiAI.cmake, so [CPU:Bugfix] Isolate KleidiAI ISA compile targets #3 remains a separate reviewable PR.
  • Verified locally with ./test.sh static, git diff --check, actionlint, the complete ./test.sh local host suite (7 passed), the PyMNN unit suite (180 passed, 4 skipped), and an arm64 iOS framework build. GitHub Android and iOS builds pass on their hosted runners.

Module

Infra

Type

  • Feature
  • Bugfix
  • Perf
  • Refact
  • Style
  • Doc
  • Test
  • Chore

Checklist

  • Commit message follows [Module:Type] Description format
  • Code compiles without errors
  • Tested on relevant platform(s)
  • No unrelated format or style changes included

jxt1234 and others added 30 commits July 21, 2026 14:20
@jxt1234
Hexagon 后端【高通NPU直接编程支持】 (alibaba#4633)
e128c55 
* [Core:Refact] setKVMeta in command session path
* [Core:Feature] Support hexagon backend
@v0jiuqi
[LLM:Bugfix] Fix silent all-zero Q4 weights when quantizing large ten…
53da9e0 
…sors on MPS (alibaba#4649)

Bugfix Qwen3.5-2B respond errorly.

Discussed-in: Merge-Request 28713212 , URL: https://code.alibaba-inc.com/AliNN/AliNNPrivate/codereview/28713212
GitOrigin-RevId: 46ceea0abf6226783ef0b4d935361870a9e0a7f2
@v0jiuqi
* [Metal:Speed] LLM prefill/decode kernel + shape/RoPE infrastructure…
d92f583 
… + Metal Op Profile. (alibaba#4650)

Combined Metal LLM decode/prefill optimization work plus supporting
core-graph and RoPE infrastructure.

GitOrigin-RevId: b71528f0de1cf17f68628031e6e59c373227f07d
@v0jiuqi
[Metal:Speed] Fused Q4/Q8 dequant+GEMM kernel + M_TILE=64 variant for…
701e1b0 
… LLM prefill on M5 (alibaba#4651)

Discussed-in: Merge-Request 28721517 , URL: https://code.alibaba-inc.com/AliNN/AliNNPrivate/codereview/28721517
GitOrigin-RevId: ae3694db47900e833e9f1dc6df59a64e0c8a7461
@v0jiuqi
[Metal:Bugfix] prefill_qkv_tensor missing ATTENTION_C4 output branch (a…
4cbcaf4 
…libaba#4654)

GitOrigin-RevId: 6712b454c35014a8b63b7126bf7635695e1b5dc5
@wangzhaode
[Infra:Release] Bump version to 3.6.1 (alibaba#4655)
6db8fac 
Discussed-in: Merge-Request 28735021 , URL: https://code.alibaba-inc.com/AliNN/AliNNPrivate/codereview/28735021
GitOrigin-RevId: 396942a40165e5b46c86de124756986c7a9becad
@Fatemanx
* [CUDA:Bugfix] Fix pooling pads and CUDA TopKV2 largest flag，Reorgan…
b418a75 
…ize pooling and TopKV2 tests (alibaba#4627)
@v0jiuqi
[Tools:Feature] llm_bench: add '-fa' meaning whether to use flash att…
daae35c 
…ention(default 1, using flash attention) (alibaba#4658)

Discussed-in: Merge-Request 28749507 , URL: https://code.alibaba-inc.com/AliNN/AliNNPrivate/codereview/28749507
GitOrigin-RevId: 6f6b024d4e4c3f23a483aed5e7d0443102295a6f

Signed-off-by: jingbang.yjb <jingbang.yjb@alibaba-inc.com>
@Qxinyu
[OpenCL:Bugfix] Fix maskless attention setArg overflow (alibaba#4653)…
4235054 
…, LinearAttention OOB writes, shared runtime gpuMode clobbering, int loop staging size and shared_gather build error (alibaba#4659)
@wangzhaode
[Infra:Release] Update PyMNN packaging for 3.6.1 release (alibaba#4657)
d407447
@wangzhaode
[Infra:Bugfix] Pin PyMNN CI test dependencies (alibaba#4665)
308e251
@v0jiuqi
[LLM:Bugfix] Do not use static global variables. (alibaba#4666)
68ad2a1 
GitOrigin-RevId: e9398e831cddfe7fde41119dc731e871ebd2f2aa

Signed-off-by: jingbang.yjb <jingbang.yjb@alibaba-inc.com>
@v0jiuqi
[Metal:Bugfix] Decide flash-attn prefill before KV alloc and fail KV …
c751ba3 
…cache expansion safely (alibaba#4668)

Link: https://code.alibaba-inc.com/AliNN/AliNNPrivate/codereview/28810722
GitOrigin-RevId: 7512195623e71f808e9d804b7982b18c4b5dbb99
@v0jiuqi
[LLM:Feature] Add one-command iOS LLM benchmark and fix Metal4 tensor…
797e7d1 
… API on iOS 26.5

Link: https://code.alibaba-inc.com/AliNN/AliNNPrivate/codereview/28753059
GitOrigin-RevId: d42a51795a007dd4e3c74a2bc75afb96eaf6fff4
@huangzhengxiang
[QNN:Feature] Add intermediate tensor dumps for QNN backend debug (al…
4cdb018 
…ibaba#4663)
@Qxinyu
[CUDA:Bugfix] Fix int32 REALDIV not dispatched in BinaryBlit (alibaba…
15a4457 
…#4652)
@Qxinyu
[Feature:Bugfix] Fix pooling pads bug for metal, vulkan, trt, hexagon…
c8ef6d8 
… backend. (alibaba#4672)

Link: https://code.alibaba-inc.com/AliNN/AliNNPrivate/codereview/28739892
GitOrigin-RevId: bff114e4f25a98001b7e5615b982f34083545637
@v0jiuqi
[LLM:Bugfix] Fix converter error for talker models. (alibaba#4677)
e7a762b 
Link: https://code.alibaba-inc.com/AliNN/AliNNPrivate/codereview/28847826


GitOrigin-RevId: 0b9d0c7860d8c75619f22ccde61b0694bdff9b59

Signed-off-by: jingbang.yjb <jingbang.yjb@alibaba-inc.com>
@v0jiuqi
[Metal:Perf] LLM prefill/decode optimization batch for Apple GPUs (M4…
2502b11 
…/M5 measured) (alibaba#4678)

Link: https://code.alibaba-inc.com/AliNN/AliNNPrivate/codereview/28821326

GitOrigin-RevId: 1792d0782acc7303eeeb5482b44c8e181fbf5f6c
@Qxinyu
[OpenCL:Feature] Support prefix kvcache file (setPrefixCacheFile) (al…
b6e0e9f 
…ibaba#4680)

GitOrigin-RevId: abdaf398c36e56e97358be95dc8270639bbc02cf
@blueskycoco
[QNN:Feature] add vtmc_mb arg for vtcm size configurable (alibaba#4587)
0298710 
Npu's vtcm can't be all used by MNN, add vtcm_mb paramter in model conversion.
Signed-off-by: Dillon Min <dillonhua@gmail.com>
@v0jiuqi
[LLM:Bugfix] Export q/k norm for InternVL models with Qwen3 LLM (fix a…
5927c1b 
…libaba#4681) (alibaba#4685)

GitOrigin-RevId: b9fd107e9985af886e646cdfdbcdfb3d929744c1
@v0jiuqi
[LLM:Feature] llm_bench: -pg now runs a real prefill+decode test shar…
5bc51d0 
…ing the KV cache (llama-bench semantics) (alibaba#4692)

Link: https://code.alibaba-inc.com/AliNN/AliNNPrivate/codereview/28916935
GitOrigin-RevId: be39f501b53dccc8fa2222c38e5da6be372f0a1a
@v0jiuqi
[LLM:Bugfix] Fix Omni OpenCL crash and talker kv_cache export (alibab…
0fd383e 
…a#4693)

Link: https://code.alibaba-inc.com/AliNN/AliNNPrivate/codereview/28890381
GitOrigin-RevId: cf70cc4e3cba8ff677b49a373b2ae5ce2bde3527
@v0jiuqi
[LLM:Bugfix] Support Qwen3.5 text-only model export (alibaba#4694)
622d379 
Link: https://code.alibaba-inc.com/AliNN/AliNNPrivate/codereview/28893110
GitOrigin-RevId: c1ec6a56125e6d78a3e8c9d454d696243e09ba80
@Qxinyu
[OpenCL:Bugfix] Fix int32 Select/Reduction on OpenCL (fp16 data corru…
c14f2b3 
…ption) (alibaba#4695)

GitOrigin-RevId: c4703efad057c69f4567a0dc1df493299b14fff5
@v0jiuqi
[LLM:Bugfix] Guard LlmContext token/string fields with mutex to fix c…
c43bcf5 
…ross-thread UAF crash (alibaba#4696)

Link: https://code.alibaba-inc.com/AliNN/AliNNPrivate/codereview/28899001
GitOrigin-RevId: 7e2ea85080eb7d51629efc68241c1b263d1de970
@v0jiuqi
[Core:Bugfix] Fix 9 memory safety vulnerabilities in multiple operators
89258d1 
and Fix over-strict validation breaking legit Crop/ScatterNd cases.(alibaba#4697)

Link: https://code.alibaba-inc.com/AliNN/AliNNPrivate/codereview/28815723

GitOrigin-RevId: 44b25c041e0182dec68db1304cbaa8e98a65a2e5
@v0jiuqi
[LLM:Bugfix] Fix LoRA-split producing garbage when transformerFuseC4 …
1f99348 
…is on (alibaba#4699)

Link: https://code.alibaba-inc.com/AliNN/AliNNPrivate/codereview/28967792
GitOrigin-RevId: 4a96916d3ce6ab5b369cb4404398280395bf167c
@wangzhaode
[Hexagon:Perf] Optimize FP16 activations with PWL (alibaba#4700)
3ed5e9c 
Discussed-in: Merge-Request 28957722 , URL: https://code.alibaba-inc.com/AliNN/AliNNPrivate/codereview/28957722
GitOrigin-RevId: 78718f83d2349acfb6f02212f0f1d1bd4c53906e
wangzhaode and others added 6 commits July 31, 2026 17:39
@wangzhaode
[CPU:Perf] Optimize SpacemiT K3 RVV IME2 LLM inference (alibaba#4702)
01a9ee8 
Discussed-in: Merge-Request 28973414 , URL: https://code.alibaba-inc.com/AliNN/AliNNPrivate/codereview/28973414
GitOrigin-RevId: 7469b9e70746524683c517734c46f8da761f91ff
@v0jiuqi
[Metal:Perf] LLM Apple-GPU optimization + attention/env overhaul. (al…
e1b8a9c 
…ibaba#4703)

- Fused attention: tensor-API prefill + single-pass decode SDPA
    - Data-driven causal mask; drop MNN_METAL_QK_CAUSAL_TRI
    - GEMV/LN fusion, encode replay, arch-tiered M64 GEMM
    - Env registry cleanup; embedding scalar sentinel; pooling pads fix
Link: https://code.alibaba-inc.com/AliNN/AliNNPrivate/codereview/28976591

GitOrigin-RevId: 0538cfd58ec85db6517a8b893047192dacf45ea1
@v0jiuqi
[Metal:Perf] Optimize LinearAttention prefill and decode (alibaba#4710)
c74aaf4 
- Optimize fused gated-delta-rule shaders and fix flash-chunk prefill correctness
- Add tensor-API flash-chunk and non-tensor-API SGMM prefill paths
- Gate SGMM to dk=128 with a runtime fallback to scalar chunking
- Route short prefill through fused simdgroup kernels and enable decode replay
Link: https://code.alibaba-inc.com/AliNN/AliNNPrivate/codereview/28933952
GitOrigin-RevId: 2d56a5cf34fe1a0748e9417a853bb2cb0daaddbf
@Qxinyu
[Core:Bugfix] Fix SIGSEGV in KL PTQ when Raster has no valid indexed …
602d160 
…inputs (alibaba#4711)

GitOrigin-RevId: 79dd443e4593ee3e20de9afdcf6494d17547705e
@Qxinyu
[QNN:Feature] Add Deconvolution, fix Interp/Flatten/LeakyReLU, improv…
256a3ae 
…e error handling (alibaba#4712)

Link: https://code.alibaba-inc.com/AliNN/AliNNPrivate/codereview/28636692

GitOrigin-RevId: 050efbcd2d949f217960f9568dcca28489993ea0
@pruthvikar
[Infra:Chore] Merge upstream MNN master
45bf66b
@pruthvikar pruthvikar self-assigned this Aug 4, 2026
@github-actions

github-actions Bot commented Aug 4, 2026

Copy link
Copy Markdown

Seems you are using me but didn't get OPENAI_API_KEY seted in Variables/Secrets for this repo. you could follow readme for more information

@pruthvikar pruthvikar removed their assignment Aug 4, 2026
@github-actions

github-actions Bot commented Aug 4, 2026

Copy link
Copy Markdown

Seems you are using me but didn't get OPENAI_API_KEY seted in Variables/Secrets for this repo. you could follow readme for more information

@pruthvikar
pruthvikar marked this pull request as ready for review August 4, 2026 09:01
@pruthvikar
pruthvikar merged commit ff8f1b8 into cleanup/remove-unused-apps-and-projects Aug 6, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

8 participants

@pruthvikar @jxt1234 @v0jiuqi @wangzhaode @Fatemanx @Qxinyu @huangzhengxiang @blueskycoco