Repository navigation
torch-vulkan: a PyTorch backend with 300+ Slang compute kernels #14461
asemptote
started this conversation in
Show and tell
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
torch-vulkan 0.2.0 runs PyTorch on Vulkan, with all its kernels written in Slang: element-wise maths templated per dtype (including a float64 maths library and correctly rounded float32 division and sqrt), cooperative-matrix GEMM for fp16/bf16, register-resident FFTs, blocked tridiagonal reduction and divide-and-conquer
eigh, bidiagonal SVD, and sparse kernels. Every op PyTorch's own test suites exercise runs on the GPU, from an H100 down to a Mali phone with no float64 or int64. Slang's generics made the per-dtype templating, and the paths for GPUs without float64 or int64, manageable.Two things we tripped over, in case they're useful:
uint2) to a pointer goes through a 64-bit integer, which needsshaderInt64. On GPUs without it we use an inlinespirv_asmOpBitcastfrom the two words instead.lsuffix to avoid silently losing precision.Thanks for Slang. I directed the development, with Claude Opus 5.5 writing the code.
All reactions