Repository navigation
Overview
Support simplified CLI commands; faster kernels; broader customzied kernel (in python/cuda/ptx) supported and debugged.
Key Changes Since Last Release
Research
For complete three-projection training, two production runs measured 1.085–1.095× speedup:
- Total launches: 45 → 32
- Pack launches: 18 → 5
- Private workspace: about 18% larger
Other MLPs were neutral or slightly slower, so separate packing remains the default. Odd dimensions, single-row inputs, and flattened rank-three linear inputs pass training and changed-input CUDA Graph checks.
Recent changes add 15 exact-shape FP32 matrix dispatch entries, a parallel MSE reduction limited to 128×128 tensors, and a content-addressed OA bundle cache that refreshes weights and excludes reference cubins. The combined matrix/loss candidate improved complete dense FP32 training 6.47–6.88× versus the OA baseline on A100-SXM4-80GB.
Current State
- Training replay: unchanged OA kernels delivered 6.9–9.0× faster MLP host steps, 11.2–15.7× for attention, and 1.18–1.25× for the Tensor Core model.
- Ordered MSE: 1.41–1.76× faster on five measured sizes, preserving bitwise results. Regressing and unmeasured sizes retain serial MSE.
- Optional binary-only builds: 1.57–1.68× faster, with identical native cubin bytes.
- Fixed the profiler-name loophole, missing inventory rules and collection-dependent import test. Added clearer compilation and runtime timing records.
Production
- Added FP16 BMM forward/dX/dW and composed attention support: batches ≤32, matrix dimensions ≤512, including tails.
- Validated on A100. Complete training improves 1.03–1.22× over OA’s padded FP16 baseline.
- Simplified CLI, added static inspection and saved build-error reports.
- Removed unused legacy vendor execution code and updated current documentation.