Repository navigation
DAT2 transformer (4x UltraSharp V2) running fully on the Android GPU with ncnn + Vulkan #7130
MikalaiKryvusha
started this conversation in
Show and tell
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
We'd like to share a result we believe is a first for ncnn: a DAT2 window-attention transformer — Kim2091's 4x UltraSharp V2 — running entirely on an Android phone's GPU through ncnn's Vulkan backend. It ships in KARTUN, an open-source (MIT) on-device image upscaler for Android; release 1.0 is out.
Result on a Mali-G615 (MediaTek MT8792, Android 16): the float output matches the original PyTorch network at 66.58 dB PSNR (largest difference 0.0021), every layer on the GPU; a 128 × 128 tile takes about 11 s.
What it took
(C, H, W) → view(C·H/h, h, W/w, w) → permute(0, 2, 1, 3) → view(C, nW, h·w) → permute(1, 2, 0), and the reverse mirrors it.torch.rollover two axes became two single-axis rolls,F.normalizebecameq / ‖q‖. Every step was checked against the original network in PyTorch to 1e-5 before pnnx.int(shape)) becomes constant at a fixed 128 px input, so pnnx traces a single graph; the engine pads edge tiles to that size.sdpa_cross.comp: it computes 4×4 blocks and bound-checks its writes, its loads are unchecked. Channel attention has 30 rows, not a multiple of 4, so the last block of the last head reads 128 KB past the buffer; desktop GPUs tolerate it, Mali leaves the output unwritten. Reported in Vulkan SDPA: sdpa_cross.comp reads past the buffer when seqlen % 4 != 0, output wrong on Mali-G615 #7120; the fix (bound-checked loads) lives in our fork, MikalaiKryvusha/ncnn20260526-kartun.1, commit e26eef0. A bisect showed thata2d60e2b(the GLSL macro move) hides the symptom in master; the unchecked read is still there.Links
tools/ncnn_dat.py(the DAT rewrite) andtools/convert-model.py --fixed 128(pnnx + a PSNR check against PyTorch)researches/08_dat2_on_ncnn.mdThe same window partition is what Swin, SwinIR and HAT use, so the approach should carry over to them (#4299). Thanks to nihui and everyone behind pnnx and the Vulkan SDPA layer — this was built on them.
All reactions