Skip to content

Releases

torch-vulkan v0.2.0

Choose a tag to compare

@github-actions github-actions released this 12 Nov 06:08

torch-vulkan 0.2.0

The first public release: a native PyTorch device ("vk") on portable Vulkan
compute, aimed at scientific workloads (float64, complex numbers, FFTs, dense
and sparse linear algebra, special functions) on integrated, discrete and
phone GPUs, without vendor libraries.

Highlights

  • No CPU fallbacks in PyTorch's tests. Across 36 of PyTorch's
    device-generic test files (about 160,000 tests each on lavapipe and an NVIDIA
    GPU), no op computes on the CPU or calls it to raise an error; every
    remaining failure is shared with the CPU or listed with its reason. Every
    OpInfo op runs on the GPU and matches the CPU.
  • Results that match the CPU, independent of the driver: Cephes-style
    float32 transcendentals, a float64 math library, correctly rounded float32
    division, sqrt and rsqrt and float64 division without fused multiply-adds, exact float8 conversions, and fixes
    for driver quirks found on Intel (Windows and Mesa), NVIDIA, Arm Mali and
    lavapipe.
  • Faster kernels: subgroup-tiled matrix products (float32 level with MKL on
    an Intel UHD 620), float16 on tensor cores through cooperative matrices
    and bfloat16 on tensor cores (155-165 TFLOP/s on an H100 NVL or L40S, about
    95% of NVIDIA's own Vulkan sample on the H100), FFTs in register-resident passes
    with the real-to-complex step fused in (within 2x of CUDA, except
    transforms of 2^16-2^20 points at up to about 2.3x, from per-call latency),
    a matrix-vector kernel, eigh by blocked tridiagonal reduction and divide
    and conquer (float64 at 1.5-3.4x CUDA's time, from 50-110x), svd by
    bidiagonalisation (0.2-0.8x: faster than CUDA in all our measurements), and
    solve and qr with their panels spread over many workgroups (solve
    1.4-3.4x; qr of a 1000x64 matrix about 4-5x, from 20-40x).
  • Training: fused and foreach Adam, AdamW, SGD and Adagrad, and
    torch.amp.GradScaler, all on the GPU.
  • New coverage: quantisation (int8 weight-only and dynamic 4-bit matrix
    products, float8 _scaled_mm, per-tensor affine quantised tensors, fake
    quantisation), nested tensors, nn.MultiheadAttention's fast path, Lanczos
    upsampling, complex32 and bcomplex32, segment_reduce.

Installing

Wheels for Windows and Linux (x86_64), Python 3.10 or newer, PyTorch 2.14:

pip install "torch==2.14.*"
pip install <wheel URL from the assets below>

On Android (Termux), build from source with the prebuilt kernels: extract
torch-vulkan-spirv-v0.2.0.tar.gz and run tools/build-termux.sh with that
directory.

Tested on

Intel UHD Graphics 620 (Windows), Intel HD Graphics 530 (Mesa ANV, Linux),
NVIDIA H100 NVL and L40S (Linux), Arm Mali-G57 (Android, Termux), Mesa
lavapipe (CI, with the validation layer).

Known limitations

torch.compile has no fusing backend yet (planned for 0.3.0). One device;
tensors under 2^31 elements;
per-tensor affine quantisation only. See the README's Limitations section and
benchmarks against CUDA.

The development was directed by Asem Wardak, with the code written by Claude
Opus 5.5 (Anthropic); every change is checked against PyTorch's CPU results.