Skip to content

v0.3.1

Latest

Choose a tag to compare

@lucialuka lucialuka released this 14 Nov 07:12
· 19 commits to main since this release

Overview

Support simplified CLI commands; faster kernels; broader customzied kernel (in python/cuda/ptx) supported and debugged.

Key Changes Since Last Release

Research

For complete three-projection training, two production runs measured 1.085–1.095× speedup:

  • Total launches: 45 → 32
  • Pack launches: 18 → 5
  • Private workspace: about 18% larger

Other MLPs were neutral or slightly slower, so separate packing remains the default. Odd dimensions, single-row inputs, and flattened rank-three linear inputs pass training and changed-input CUDA Graph checks.
Recent changes add 15 exact-shape FP32 matrix dispatch entries, a parallel MSE reduction limited to 128×128 tensors, and a content-addressed OA bundle cache that refreshes weights and excludes reference cubins. The combined matrix/loss candidate improved complete dense FP32 training 6.47–6.88× versus the OA baseline on A100-SXM4-80GB.

Current State

  • Training replay: unchanged OA kernels delivered 6.9–9.0× faster MLP host steps, 11.2–15.7× for attention, and 1.18–1.25× for the Tensor Core model.
  • Ordered MSE: 1.41–1.76× faster on five measured sizes, preserving bitwise results. Regressing and unmeasured sizes retain serial MSE.
  • Optional binary-only builds: 1.57–1.68× faster, with identical native cubin bytes.
  • Fixed the profiler-name loophole, missing inventory rules and collection-dependent import test. Added clearer compilation and runtime timing records.

Production

  • Added FP16 BMM forward/dX/dW and composed attention support: batches ≤32, matrix dimensions ≤512, including tails.
  • Validated on A100. Complete training improves 1.03–1.22× over OA’s padded FP16 baseline.
  • Simplified CLI, added static inspection and saved build-error reports.
  • Removed unused legacy vendor execution code and updated current documentation.