Precision Capability Matrix

Precision Capability Matrix

What each precision datapath implements, and — the part that matters — which blanks are real.

Generated from the source tree by tools/precision_matrix.sh (make matrix); the same table is in README.md. Function counts: i8=167 f16=73 i4=74 i16=22.

capability	i8	f16	i4	i16
regcmd synth	✅	✅	✅	✅
output stage (requant)	✅	✅	—	✅
fused-act output stage	✅	—	—	—
pack weights	✅	✅	✅	—
load / .orkpack persist	✅	✅	✅	—
zero-copy import / adopt	✅	✅	✅	—
quantise from f32	✅	—	✅	—
run — single core	✅	✅	✅	—
run — multicore	✅	✅†	✅	—
run — HW chain	✅	—	✅	✅†
run — async stream	✅	✅	✅	—
run — NONBLOCK doorbell	✅	✅†	✅	—
batched GEMM (bmm)	✅	✅	✅	—
fused matmul+activation	✅	✅	—	—
SDP activations	✅	—	—	✅
elementwise mul	✅	✅	—	✅
elementwise add	✅	✅	—	✅
per-channel multiply	✅	✅	—	✅
slice-and-dice tiles	✅	—	✅	—
M-fold chain	✅	—	—	—
MoE expert coalescing	—	—	✅	—
resident KV	✅	—	—	—
probes / RE replay	✅	✅	✅	✅
regcmd fuzz hooks	✅	✅	✅	—

† f16 / run — multicore — i8/colsplit.c is the ONLY fp16 multicore path (#45) (ork_dyn_begin_colsplit) † f16 / run — NONBLOCK doorbell — same colsplit path — fp16 wide-K rides the doorbell (ork_dyn_begin_colsplit) † i16 / run — HW chain — the i16 chain rides the general PC-chain core in i8/probe.c rather than a dedicated i16 one (ork_npu_chain_progs)

Reading the blanks

The 179-vs-22 function gap between int8 and int16 reads like a huge hole. It is mostly not. Sorting the blanks by why:

By design — the dtype is not that kind of thing (no work to do)

blank	why
int4 — every SDP/activation row (fused-act stage, fused matmul+activation, SDP activations, elementwise mul/add, per-channel multiply, requant output stage)	RK3588 is W8A8 or W4A4 symmetric; W4A8 is not a hardware mode. int4 activations are int4, and the SDP LUT op consumes int8/int16 — so there is no int4 activation path to write. See npu-int4-datapath-symmetric and the Exp log.
int16 — pack / load / import / quantise from f32 / slice / fold / KV	int16 is the activation precision, the accuracy tier between int8 and fp16 for SDP ops. It is not a weight-storage tier: nothing packs int16 weights, so none of the weight-side machinery applies.
fp16 — quantise from f32	fp16 is not a quantised tier: f32 → fp16 is a cast, not a fit, so there is no quantiser to write. Persist and zero-copy import are no longer blanks — see "Corrected 2026-08-30" below.
int8 — MoE expert coalescing	experts are packed NF4/int4 by the auto-profile, so the coalesced-expert path is int4's by construction.

Genuinely open — someone could implement these

blank	note
fp16 — HW chain	fp16 rides the doorbell colsplit (†) but has no PC-chained multi-task graph like run_chain_i8. Whether it would pay is unmeasured.
fp16 — slice-and-dice tiles	the c_base tile decomposition is int8/int4 only; fp16 wide shapes go through colsplit instead.
int4 — resident KV, M-fold	both are int8-only optimisations for the production path. Neither has been tried at int4.
int16 — multicore, async stream, bmm	plausible but unmotivated: int16 exists for SDP accuracy, and no caller wants a streamed or core-split int16 matmul today. (The multicore cell read ✅† until check 8 asked it to name a symbol and it could not — there is no i16 multicore path. The only i16 × multicore thing in the tree is ORK_OP_MATMUL_I16OUT_I8, an int8 matmul that emits int16, which is the int8 row.)

Not a gap: shared implementations (†)

A dagger means the capability exists for that precision but is provided by another module's code — fp16's multicore/doorbell path is ork_dyn_begin_colsplit in i8/colsplit.c, and int16's HW chain rides the general PC-chain core ork_npu_chain_progs in i8/probe.c. Counting those as int8 lines is a real part of why int8 looks disproportionately large.

Keeping it honest

The table is generated, so it tracks the code. What it cannot see is a shared implementation serving a precision under another dtype's name. Those are asserted by hand in tools/precision_overrides.tsv.

Hand-asserted claims rot, so each row must cite the symbol doing the work, and make check-registry check 8 fails the build if that symbol stops being defined, if the capability is not one the matrix emits, or if the dtype is not a real column. A dagger therefore cannot outlive the code it points at.

That gate paid for itself on its first run: it deleted a run — multicore / i16 row, because asked to name the symbol behind "routed through the int8 chain", there was none — no i16 multicore path exists.

One direction stays unguarded, and knowing which matters. If someone makes a precision reuse another's implementation and adds no row, the table silently under-reports that dtype, and nothing detects it — there is no ground truth to diff a name-based classifier against. The only defence is the checklist line in AGENTS.md §"Documentation review on every commit". If the matrix is ever load-bearing rather than orientation, the fix is not better names (they cannot express "one function, two dtypes" at all) but probe-anchoring it like OPS_REGISTRY.md: a cell is ✅ because a test drives that dtype through the public API, which is blind to which file implements it and claims "proven on hardware" instead of "a symbol exists".

---

Corrected 2026-08-30 — fp16 persist and import are no longer blanks

This page previously carried the blanks as by design:

fp16 — .orkpack persist, zero-copy import, quantise from f32 — the on-disk pack format is int8/int4. fp16 weights are transient (attention operands, dynamic GEMM), not resident, so there is nothing to persist or import.

The last clause was wrong, and wrong in the direction this page warns about: it explained a gap as a design decision when it was a missing implementation. fp16 weights are packed NPU-resident tiles exactly like int8 ones, and ork_w_dump has always been dtype-agnostic — so the write half worked and only the read-back was absent. ork-driver#1 identified it; 7d515bf closed it with ork_f16_mm_load / ork_f16_mm_load_import.

Two cells flip. The "quantise from f32" blank stays, and its real reason is narrower than the old note: f32 → fp16 is a cast, not a fit, so there is no quantiser to write.

One cell is now over-reported, and the classifier cannot fix it

zero-copy import / adopt reads ✅ for f16, but that row conflates two different things:

	dma-buf import (*_load_import)	daemon adopt (ork_i8_mm_import → ORKD_IMPORT)
i8	✅	✅
f16	✅	—
i4	✅	—

fp16 has the import; it does not have the daemon-routed adopt, because ORKD_IMPORT is int8-hardcoded end to end (wire struct, orkd_handle_import, orkd_import_i8). Supporting fp16 there is an orkd protocol change, not a driver addition. int4 is in the same position, so the row over-reports for both — it always did, and adding fp16 to it just made that visible.

The generator classifies by the dtype token in a symbol name, so it cannot split a row whose two halves share a naming pattern. This is precisely the failure the page's own closing note describes: "the fix is not better names (they cannot express 'one function, two dtypes' at all) but probe-anchoring it like OPS_REGISTRY.md." Recorded here rather than silently accepted.

Verification

test_f16_load (in make test) asserts the property that matters for a persist format: a loaded weight computes bit-identical output to the packed weight it came from — exact equality, not a tolerance, since both run the same kernel on the same tiles. Shapes are chosen so a plausible-but-wrong un-tiler diverges rather than merely degrading:

M=8 K=512   N=64     blob=    64 KB : load bit-exact, load_import bit-exact   single tile
M=8 K=4096  N=64     blob=   512 KB : load bit-exact, load_import bit-exact   K-slice (Sk=2)
M=8 K=512   N=16384  blob= 16384 KB : load bit-exact, load_import bit-exact   wide-N (Sn=2, Sn-major walk)
M=8 K=1568  N=80     blob=   248 KB : load bit-exact, load_import bit-exact   non-pow2 K, partial page pad

It also checks that a wrong-size blob is refused rather than mis-sliced — the failure mode a size-tolerant loader would turn into silent garbage.