Skip to content

Releases

v0.3.1

Choose a tag to compare

@lucialuka lucialuka released this 14 Nov 07:12

Overview

Support simplified CLI commands; faster kernels; broader customzied kernel (in python/cuda/ptx) supported and debugged.

Key Changes Since Last Release

Research

For complete three-projection training, two production runs measured 1.085–1.095× speedup:

  • Total launches: 45 → 32
  • Pack launches: 18 → 5
  • Private workspace: about 18% larger

Other MLPs were neutral or slightly slower, so separate packing remains the default. Odd dimensions, single-row inputs, and flattened rank-three linear inputs pass training and changed-input CUDA Graph checks.
Recent changes add 15 exact-shape FP32 matrix dispatch entries, a parallel MSE reduction limited to 128×128 tensors, and a content-addressed OA bundle cache that refreshes weights and excludes reference cubins. The combined matrix/loss candidate improved complete dense FP32 training 6.47–6.88× versus the OA baseline on A100-SXM4-80GB.

Current State

  • Training replay: unchanged OA kernels delivered 6.9–9.0× faster MLP host steps, 11.2–15.7× for attention, and 1.18–1.25× for the Tensor Core model.
  • Ordered MSE: 1.41–1.76× faster on five measured sizes, preserving bitwise results. Regressing and unmeasured sizes retain serial MSE.
  • Optional binary-only builds: 1.57–1.68× faster, with identical native cubin bytes.
  • Fixed the profiler-name loophole, missing inventory rules and collection-dependent import test. Added clearer compilation and runtime timing records.

Production

  • Added FP16 BMM forward/dX/dW and composed attention support: batches ≤32, matrix dimensions ≤512, including tails.
  • Validated on A100. Complete training improves 1.03–1.22× over OA’s padded FP16 baseline.
  • Simplified CLI, added static inspection and saved build-error reports.
  • Removed unused legacy vendor execution code and updated current documentation.

v0.3.0

Choose a tag to compare

@lucialuka lucialuka released this 08 Nov 09:54

Kernel Research

Main Results

  • The full retained control corpus was replayed on an A100. It covered 42 scalar producer/consumer dependency families plus 14 tensor probes. The broader probe generator exposes 541 standalone SM80 PTX forms; those are recorded as inventory-only because a standalone opcode does not define a meaningful dependency edge. The measured minima are mostly S4/S5 for fixed-pipe RAW dependencies, S13 for predicate-to-guard cases, and S1/S2 for the tested store/address cases.
  • The CUTLASS-style load-next/compute-current candidate is implemented. It alternates two fragment register windows, loads the next K fragment between current HMMA groups, preserves the existing async-copy/barrier protocol, and enforces accumulator K-order.

Shape Specific Kernel Optimization

It improved the OA schedule on five shapes:

  • 512×512×2048: about 2.4–2.5% faster.
  • 1024×4096×1024: about 8.2–8.3% faster.
  • 2048×512×512: about 1.8% faster.
  • 4096×1024×1024: about 9.1–9.2% faster.
  • 128×128×128: about 0.7–1.3% faster.
    It regressed 4096×4096×4096 by about 9.8–9.9% and increased register use by 24 registers per thread. The candidate remains shape-specific and experiment-only. Matched ptxas remains faster.
  • Grouped-N CTA traversal was tested against m-fast and Morton-8. All numerical and resource gates passed.
    • 16384×4096×4096: grouped-N improved OA by roughly 6–7% over m-fast and was slightly ahead of Morton.
    • 1024×4096×1024: no stable grouped-N win.
    • 4096×4096×4096: grouped-N was effectively neutral or slightly slower.
      The exact measured results are captured in measured_dispatch_manifest.json. Only exact FP16, 16-byte-aligned, zero-workspace cases are classified as validated-experiment-only; neighboring alignment, workspace, BF16 and shape variants do not inherit the result.
  • The lowered dependency scheduler now runs over actual OA-lowered GEMM blocks in actual_block_schedule.py. HMMA-bearing blocks are accepted only when every physical accumulator group retains K-order. Terminator blocks fail closed. The resulting source-order permutations are recorded in actual-block-schedule.json. No reordered SASS stream has been encoded or promoted.

Engineering CLI

Applied the native-first code changes and added a measured runtime optimization to the production compiler.

Main Production Changes

  • Independent native source files now compile through a bounded ThreadPoolExecutor.
  • Default is four workers, capped at eight.
  • Set OPENASSEMBLER_BUILD_JOBS=1 to restore serial behavior.
  • --dump-ptx stays serial so PTX output remains deterministic and readable.
  • executor.map preserves source order.
  • Each source still passes through the same backend, verification, ptxas, disassembly and manifest gates.
  • Cubin, SASS and manifest publication remains serialized and deterministic.

Compilation Measurements

  • examples/: 5 sources, 14 kernels
    • Serial: 28.42 seconds
    • Four workers: 22.81 seconds
    • Improvement: 1.25×
    • Cubins and manifests: byte-identical
  • Whole repository native scan: 26 candidate sources, 21 compiled entries, 61 OpenAssembler kernels
    • Serial: 29.18 seconds
    • Four workers: 19.62 seconds
    • Improvement: 1.49×
    • Output trees and manifests: byte-identical
    • Five files still fail because they use unsupported constructs such as ld.global.v2, bar.warp.sync, or unsupported FP32 modifiers. These remain explicit failures rather than silent fallbacks.

Native-default Frontend Behavior Validation:

  • The following command now fails clearly because the repository has no native kernels and application capture was not explicitly requested
openassembler compile testcases/test_whole_repo --dry-run 
  • The following command correctly selects the application frontend.
openassembler compile testcases/test_whole_repo --frontend torch --backend modal --verify --dry-run
  • A full explicit Torch capture/replay completed on the A100 with:
    • 2 captured graphs
    • 6 optimizer steps
    • 0 new GPU code generation during replay
    • 77 tensors checked
    • verification status: ok

v0.2.1

Choose a tag to compare

@lucialuka lucialuka released this 05 Nov 15:18

1. Key Researches

Improve OpenAssembler compiler's speed by 10-12% in MMA operations and ReLU.
OpenAssembler compiler architecture:

NVIDIA CUDA complete list of compilers architecture:

Implemented and validated the current CLI/training pipeline changes.

2. Key changes:

  • Added explicit decoder-only Transformer blocks:
    • causal self-attention
    • RMSNorm
    • MLP residual blocks
    • causal-attention regression tests
  • Added CUDA-event stage timing:
    • batch preparation
    • forward/loss
    • backward
    • optimizer
    • end-to-end step
    • tokens/sec
  • Added eager-vs-compiled timing records to application reports.
  • Added SSH transport support for PyTorch applications:
    • --build-backend ssh
    • --build-host user@host
    • bounded remote worker execution
    • temporary remote bundle cleanup
  • Added openassembler benchmark CLI matrix with timestamped JSON evidence.
  • Improved openassembler status to handle the current status-document format.
  • Updated Transformer fixture README/configuration and timing options.

3. Validation:

58 integration, runner, and CLI tests passed.

  • Python syntax checks passed.
  • Modal A100 compile succeeded:
    • 2 graphs
    • 67 generated CUDA kernels
    • 77 eager/compiled tensors verified
  • Modal cache-only replay succeeded:
    • new_gpu_codegen: 0
    • 6 optimizer steps
    • 77 tensors verified
  • A100 timing evidence recorded:
    • eager end-to-end median: 4.756 ms
    • compiled-cache end-to-end median: 3.120 ms
    • eager throughput: 6,728 tokens/s
    • compiled throughput: 10,257 tokens/s

v0.2.0

Choose a tag to compare

@lucialuka lucialuka released this 03 Nov 01:05
941813c
openassembler_logo

What it include

  • A SASS encoder/decoder for sm_80 (Ampere) — opcodes, operand forms, control bits (stall, yield, scoreboards, wait masks) — recovered by bit-flip fuzzing against nvdisasm (docs/findings-sm80-encoding.md).

  • A Python / CUDA C/C++ / PTX → SASS compiler (register allocation, scheduling, 4× loop unrolling). A historical 203-probe run emitted 2,132 instructions versus ptxas's 2,249 (-5.2%). Selected microbenchmarks ranged from 0.92× to 1.36× ptxas speed; NN/ transformer baselines and their numerical mismatches are reported separately.

  • A functional SASS emulator, fitted to real A100 behaviour: tensor-core (HMMA/IMMA/DMMA) numerics bit-exact on 512/512 recorded lanes, instruction latencies measured directly on hardware (not guessed from nvdisasm's mined minimum distances, which are provably wrong — see below).

  • Division/remainder by a constant, replacing ptxas's always-runtime reciprocal sequence with a [z3-proven](openassembler/ opt/divconst.py) multiply-high lowering: 13 → 2–3 instructions.

  • Strassen / AlphaTensor fast matrix multiplication: 97 published low-rank factorizations of matrix multiply (AlphaTensor, DeepMind/Nature 2022) imported, verified exactly over the rationals, compiled by our backend, and measured against cuBLAS on an A100 at n = 8192.

  • Python and CUDA C/C++ frontends: write a kernel in a Python subset (openassembler/pyfront/, built from scratch) or in CUDA C/C++ (lowered to PTX by nvcc), and it is compiled to SASS by our backend (docs/FRONTENDS.md).

  • openassembler, a CLI that ties all of this together: compile a PTX, Python or CUDA C file to a .cubin plus a readable .sass with our backend, verify it against a reference interpreter, and run compiled cubins against a ptxas reference on a local GPU, an SSH host, or a cloud GPU. The separate gpu run command tests the probe corpus.

New Features

  • Added float division, sqrtf, casts, integer register division/remainder.
  • Added pointer-increment and 64-bit loop support.
  • Added cross-warp shared-memory barrier modeling.
  • Added repository compilation with output/cubin, output/sass, manifests, run, and cnr.
  • Added Modal reproduction instructions and --spec support.
  • Added NN/transformer baseline study and evidence report. (mainly for reproduction when aiming for academic usages)