Skip to content

Hierarchos-Native v0.2.0 — RDNA3 WMMA

Latest

Choose a tag to compare

@necat101 necat101 released this 17 Nov 08:21
· 2 commits to main since this release

Hierarchos-Native v0.2.0 — RDNA3 WMMA

This is the first public release that ships the opt-in RDNA3 cooperative-matrix
(WMMA) path
alongside the unchanged, default FP32 runtime. It also ships the
first post-v0.1.0-rc2 Windows x64 portable package built from the released
main.

What's new

Opt-in Mixed WMMA. --wmma-mode mixed runs eligible matrix projections as
FP16 × FP16 with FP32 accumulation, keeping FP32 master weights, FP32
gradients, FP32 AdamW moments and FP32 normalisation/softmax/loss. It is off
unless you ask for it.

Scoped Full FP16. --wmma-mode full-fp16 uses a genuine FP16 cooperative
accumulator with a reduction limit of FULL_FP16_MAX_REDUCTION = 64; longer
reductions run the original FP32 kernels and increment a fallback counter. It is
admitted only for the numerically qualified 64-wide single-layer Llama profile on
the qualified adapter/driver.

Capability reporting. hierarchos-native-cli precision-info prints one JSON
object describing what the device actually negotiated (mixed_supported,
full_fp16_supported, full_fp16_qualified, full_fp16_max_reduction).

Dispatch accounting you can audit. Cooperative dispatches, phases
(forward/dW/dX), the plain-FP16 control kernel, shape/policy fallbacks and
Full FP16 reduction fallbacks are counted separately, so "requested but not
dispatched" is visible rather than implied.

The default path did not regress

FP32 was qualified against the previous main (ead6cd3 → 5d40f68) on seven
workloads, 12 alternating ABBA rounds each, 5 warm-up and 30 timed forwards per
arm, distinct hashed binaries. The statistic is the median of the twelve
per-round candidate/stable ratios:

Workload Paired ratio
gpt2-unmasked 0.9996
gpt2_wide-unmasked 1.0007
gpt2_wide-padded 0.9996
llama_wide-unmasked 0.9617
llama_wide-padded 0.9903
gpt2-wide-decode 0.9962
gpt2-wide-batch32 0.9987

All seven are inside the ±2 % parity band; the worst is 1.0007. FP32 logits
were bit-identical between the two arms in all 12 rounds of all seven
workloads.

Performance, honestly

Measurement Result
Kernel-level WMMA, large GEMMs (512×768×768 … 2048×768×2048) 1.83×–2.31× faster than the FP32 kernel
Kernel-level WMMA, below crossover (64×64×64, 16×128×32) 0.74×–0.76× — slower; the shape policy keeps FP32 there
Plain FP16 storage without matrix cores slower than FP32 at every size
End-to-end 512-row training step, Mixed vs FP32 slower: 14.0444 ms vs 12.2911 ms, ratio 0.8708 (CI 0.8517–0.8898)

Because the end-to-end training step was slower, Mixed stays off by default
and no training-step speedup is claimed in this release. Kernel ratios are not
extrapolated to a pipeline.

Requirements

  • Windows 11 x64, target triple x86_64-pc-windows-msvc.
  • A Vulkan 1.x compute driver for any GPU work.
  • No Visual C++ Redistributable: all executables are statically linked against
    the MSVC C runtime. Rust, Cargo, Python and PyTorch are not runtime
    requirements.
  • Model weights are not bundled. Point the CLI at a supported local package
    (--model-path) or use a public Hub model (--hf-model).

Hardware qualification boundary

WMMA was qualified on one device: AMD RDNA3, PCI 1002:15bf (Ryzen Z1
Extreme / ROG Ally class), driver record 00800184, with a SUBGROUP-scope
16x16x16 f16/f16→f32 cooperative tile.

Not qualified and not implied: other RDNA3 parts, RDNA1/RDNA2, Intel Gen9,
multi-GPU device groups, non-Windows targets. Unsupported devices fail closed
with the probed reason; they never silently report WMMA.

Install

Get-FileHash .\Hierarchos-Native-Windows-x64-Portable-v0.2.0.zip -Algorithm SHA256
Expand-Archive .\Hierarchos-Native-Windows-x64-Portable-v0.2.0.zip -DestinationPath .\hierarchos
cd .\hierarchos

.\bin\hierarchos-native-cli.exe doctor
.\bin\hierarchos-native-cli.exe devices
.\bin\hierarchos-native-cli.exe precision-info --device-index 0

# First inference run on a small public fixture
.\bin\hierarchos-native-cli.exe infer --hf-model hf-internal-testing/tiny-random-gpt2 `
  --prompt "Hello" --max-new-tokens 32 --no-do-sample

Enable WMMA:

.\bin\hierarchos-native-cli.exe transformer-train `
  --model-path .\model --train .\data.jsonl --out-dir .\out `
  --epochs 1 --batch-size 1 --seq-len 128 --lr 5e-5 --device-index 0 `
  --wmma-mode mixed

See WMMA_GUIDE.md
for the full guide: modes, aliases, environment variable, fallback semantics,
numerical safety and limitations.

Assets

Asset Contents
Hierarchos-Native-Windows-x64-Portable-v0.2.0.zip bin\ (CLI, GUI, four Vulkan companions, hierarchos-infer), shaders, licenses, notices, BUILD_INFO.txt, DEPENDENCIES.txt, QUICKSTART.txt
SHA256SUMS SHA-256 of every attached asset
BUILD_MANIFEST.json Source commit, target triple, toolchain, build flags

Known issues and limitations

  • Mixed end-to-end training is slower than FP32 on the measured fixture; it is
    opt-in for a reason.
  • Full FP16 is experimental and narrow: one Llama-64 profile plus a reduction cap
    of 64, on the qualified adapter/driver only.
  • At released wide-multimodal and native V4.1 geometry the profitability floor
    (WMMA_MIN_PROFITABLE_MACS = 16,000,000) declines every projection, so those
    receipts record zero cooperative dispatches and qualify the FP32 route.
  • 194 of 1,116 strict intermediate diagnostics remain failures of that additional
    internal-tensor check (186 vision-feature, 8 projected-gradient). They are
    disclosed, not relabelled; the maintained max_abs < 2e-7 logit gate is
    unchanged.
  • No physical RDNA3 CI runner is registered, so the RDNA3 receipts are
    source-bound local qualifications rather than a hardware-CI pass.
  • Tolerances (mixed 2e-2/3e-3, Full FP16 5e-2/1e-2) are fixture measurements,
    not a warranty for arbitrary checkpoints.

Verification

Release receipt 90281546f4ece2332c2365e1f3068486a7b33ea0239446dbbbf681f8c2f21c19,
status PASS_SCOPED_RELEASE, source revision 2ef9b8d (2,004 bound sources,
32 evidence files, 33 binaries, 24 components). Strict evidence integrity reports
0 failures and 0 missing; the unified release suite passes 92/92 with 0 skipped.