Repository navigation
Releases
v0.3.1
Overview
Support simplified CLI commands; faster kernels; broader customzied kernel (in python/cuda/ptx) supported and debugged.
Key Changes Since Last Release
Research
For complete three-projection training, two production runs measured 1.085–1.095× speedup:
- Total launches: 45 → 32
- Pack launches: 18 → 5
- Private workspace: about 18% larger
Other MLPs were neutral or slightly slower, so separate packing remains the default. Odd dimensions, single-row inputs, and flattened rank-three linear inputs pass training and changed-input CUDA Graph checks.
Recent changes add 15 exact-shape FP32 matrix dispatch entries, a parallel MSE reduction limited to 128×128 tensors, and a content-addressed OA bundle cache that refreshes weights and excludes reference cubins. The combined matrix/loss candidate improved complete dense FP32 training 6.47–6.88× versus the OA baseline on A100-SXM4-80GB.
Current State
- Training replay: unchanged OA kernels delivered 6.9–9.0× faster MLP host steps, 11.2–15.7× for attention, and 1.18–1.25× for the Tensor Core model.
- Ordered MSE: 1.41–1.76× faster on five measured sizes, preserving bitwise results. Regressing and unmeasured sizes retain serial MSE.
- Optional binary-only builds: 1.57–1.68× faster, with identical native cubin bytes.
- Fixed the profiler-name loophole, missing inventory rules and collection-dependent import test. Added clearer compilation and runtime timing records.
Production
- Added FP16 BMM forward/dX/dW and composed attention support: batches ≤32, matrix dimensions ≤512, including tails.
- Validated on A100. Complete training improves 1.03–1.22× over OA’s padded FP16 baseline.
- Simplified CLI, added static inspection and saved build-error reports.
- Removed unused legacy vendor execution code and updated current documentation.
v0.3.0
Kernel Research
Main Results
- The full retained control corpus was replayed on an A100. It covered 42 scalar producer/consumer dependency families plus 14 tensor probes. The broader probe generator exposes 541 standalone SM80 PTX forms; those are recorded as inventory-only because a standalone opcode does not define a meaningful dependency edge. The measured minima are mostly S4/S5 for fixed-pipe RAW dependencies, S13 for predicate-to-guard cases, and S1/S2 for the tested store/address cases.
- The CUTLASS-style load-next/compute-current candidate is implemented. It alternates two fragment register windows, loads the next K fragment between current HMMA groups, preserves the existing async-copy/barrier protocol, and enforces accumulator K-order.
Shape Specific Kernel Optimization
It improved the OA schedule on five shapes:
- 512×512×2048: about 2.4–2.5% faster.
- 1024×4096×1024: about 8.2–8.3% faster.
- 2048×512×512: about 1.8% faster.
- 4096×1024×1024: about 9.1–9.2% faster.
- 128×128×128: about 0.7–1.3% faster.
It regressed 4096×4096×4096 by about 9.8–9.9% and increased register use by 24 registers per thread. The candidate remains shape-specific and experiment-only. Matched ptxas remains faster. - Grouped-N CTA traversal was tested against m-fast and Morton-8. All numerical and resource gates passed.
- 16384×4096×4096: grouped-N improved OA by roughly 6–7% over m-fast and was slightly ahead of Morton.
- 1024×4096×1024: no stable grouped-N win.
- 4096×4096×4096: grouped-N was effectively neutral or slightly slower.
The exact measured results are captured in measured_dispatch_manifest.json. Only exact FP16, 16-byte-aligned, zero-workspace cases are classified as validated-experiment-only; neighboring alignment, workspace, BF16 and shape variants do not inherit the result.
- The lowered dependency scheduler now runs over actual OA-lowered GEMM blocks in actual_block_schedule.py. HMMA-bearing blocks are accepted only when every physical accumulator group retains K-order. Terminator blocks fail closed. The resulting source-order permutations are recorded in actual-block-schedule.json. No reordered SASS stream has been encoded or promoted.
Engineering CLI
Applied the native-first code changes and added a measured runtime optimization to the production compiler.
Main Production Changes
- Independent native source files now compile through a bounded ThreadPoolExecutor.
- Default is four workers, capped at eight.
- Set OPENASSEMBLER_BUILD_JOBS=1 to restore serial behavior.
- --dump-ptx stays serial so PTX output remains deterministic and readable.
- executor.map preserves source order.
- Each source still passes through the same backend, verification, ptxas, disassembly and manifest gates.
- Cubin, SASS and manifest publication remains serialized and deterministic.
Compilation Measurements
- examples/: 5 sources, 14 kernels
- Serial: 28.42 seconds
- Four workers: 22.81 seconds
- Improvement: 1.25×
- Cubins and manifests: byte-identical
- Whole repository native scan: 26 candidate sources, 21 compiled entries, 61 OpenAssembler kernels
- Serial: 29.18 seconds
- Four workers: 19.62 seconds
- Improvement: 1.49×
- Output trees and manifests: byte-identical
- Five files still fail because they use unsupported constructs such as ld.global.v2, bar.warp.sync, or unsupported FP32 modifiers. These remain explicit failures rather than silent fallbacks.
Native-default Frontend Behavior Validation:
- The following command now fails clearly because the repository has no native kernels and application capture was not explicitly requested
openassembler compile testcases/test_whole_repo --dry-run - The following command correctly selects the application frontend.
openassembler compile testcases/test_whole_repo --frontend torch --backend modal --verify --dry-run- A full explicit Torch capture/replay completed on the A100 with:
- 2 captured graphs
- 6 optimizer steps
- 0 new GPU code generation during replay
- 77 tensors checked
- verification status: ok
v0.2.1
1. Key Researches
Improve OpenAssembler compiler's speed by 10-12% in MMA operations and ReLU.
OpenAssembler compiler architecture:
Implemented and validated the current CLI/training pipeline changes.
2. Key changes:
- Added explicit decoder-only Transformer blocks:
- causal self-attention
- RMSNorm
- MLP residual blocks
- causal-attention regression tests
- Added CUDA-event stage timing:
- batch preparation
- forward/loss
- backward
- optimizer
- end-to-end step
- tokens/sec
- Added eager-vs-compiled timing records to application reports.
- Added SSH transport support for PyTorch applications:
- --build-backend ssh
- --build-host user@host
- bounded remote worker execution
- temporary remote bundle cleanup
- Added openassembler benchmark CLI matrix with timestamped JSON evidence.
- Improved openassembler status to handle the current status-document format.
- Updated Transformer fixture README/configuration and timing options.
3. Validation:
58 integration, runner, and CLI tests passed.
- Python syntax checks passed.
- Modal A100 compile succeeded:
- 2 graphs
- 67 generated CUDA kernels
- 77 eager/compiled tensors verified
- Modal cache-only replay succeeded:
- new_gpu_codegen: 0
- 6 optimizer steps
- 77 tensors verified
- A100 timing evidence recorded:
- eager end-to-end median: 4.756 ms
- compiled-cache end-to-end median: 3.120 ms
- eager throughput: 6,728 tokens/s
- compiled throughput: 10,257 tokens/s
v0.2.0
What it include
-
A SASS encoder/decoder for sm_80 (Ampere) — opcodes, operand forms, control bits (stall, yield, scoreboards, wait masks) — recovered by bit-flip fuzzing against
nvdisasm(docs/findings-sm80-encoding.md). -
A Python / CUDA C/C++ / PTX → SASS compiler (register allocation, scheduling, 4× loop unrolling). A historical 203-probe run emitted 2,132 instructions versus ptxas's 2,249 (-5.2%). Selected microbenchmarks ranged from 0.92× to 1.36× ptxas speed; NN/ transformer baselines and their numerical mismatches are reported separately.
-
A functional SASS emulator, fitted to real A100 behaviour: tensor-core (HMMA/IMMA/DMMA) numerics bit-exact on 512/512 recorded lanes, instruction latencies measured directly on hardware (not guessed from
nvdisasm's mined minimum distances, which are provably wrong — see below). -
Division/remainder by a constant, replacing
ptxas's always-runtime reciprocal sequence with a [z3-proven](openassembler/ opt/divconst.py) multiply-high lowering: 13 → 2–3 instructions. -
Strassen / AlphaTensor fast matrix multiplication: 97 published low-rank factorizations of matrix multiply (AlphaTensor, DeepMind/Nature 2022) imported, verified exactly over the rationals, compiled by our backend, and measured against cuBLAS on an A100 at n = 8192.
-
Python and CUDA C/C++ frontends: write a kernel in a Python subset (
openassembler/pyfront/, built from scratch) or in CUDA C/C++ (lowered to PTX bynvcc), and it is compiled to SASS by our backend (docs/FRONTENDS.md). -
openassembler, a CLI that ties all of this together: compile a PTX, Python or CUDA C file to a.cubinplus a readable.sasswith our backend, verify it against a reference interpreter, and run compiled cubins against a ptxas reference on a local GPU, an SSH host, or a cloud GPU. The separategpu runcommand tests the probe corpus.
New Features
- Added float division, sqrtf, casts, integer register division/remainder.
- Added pointer-increment and 64-bit loop support.
- Added cross-warp shared-memory barrier modeling.
- Added repository compilation with output/cubin, output/sass, manifests, run, and cnr.
- Added Modal reproduction instructions and --spec support.
- Added NN/transformer baseline study and evidence report. (mainly for reproduction when aiming for academic usages)


