Index the last two days' pages — four of them were orphans reachable only by guessing the filename
Where the int4 time actually goes: NPU dispatch 72 percent, and two instrumentation lies
The DMA accounting compared non-fungible quantities. Measured attribution puts
NPU execution at 72-73% and host regcmd synthesis at 11-14% -- four to eight
times the nibble-pack that section 9 credited with deciding the transposition
A/B. Neither operand's DMA sets the clock: weight bandwidth is 0.55-0.85 GB/s
against a ~30 GB/s ceiling.
Records two pieces of instrumentation that mislead. The grow-only scratch also
has fixed pre-allocation floors, so even a single-shape process over-syncs
2.4-3.0x and orki_bsync_live is worth 8.3% at the geometry worth shipping.
ORK_DOM_PROFILE starts its timer after the drain barrier and so reports
0.97 us/swap against a real ~300 us.
Keeps the open tension on section 9's attribution rather than smoothing it:
that A/B had matched program counts, so program count cannot explain it.
Transposition is refuted: the operand roles are not symmetric in software
Swapping the matmul so the small matrix holds the weight role is expressible
and measurably slower -- 3106 us untransposed against 3642 us swapped at
matched split, Wb, programs and cores, while moving half the bytes. These
shapes are bound by neither DMA nor program count; the weight role is packed
once while the activation role is nibble-packed on the host every call, so
transposing raises per-call host work 16x.
Confirms in the same run that the Wb WRONG boundary vanishes with reuse off,
so the ceiling is weight-reuse staleness rather than containment. Records two
hazards: the WRONG points were the fastest in the sweep, and geometry validity
depends on core count, so a Wb rule keyed on K alone is unsound.
Wb can be raised 2.21x at K<=2048, but the pair is H, not the bank count
Measures the three predictions section 7 left open. Two are refuted: n(12-n)
is the wrong objective (the binding term is activation DMA, a function of Wb
alone), and transposition is net negative (the weight already streams once,
and the core clamp would cap a transposed prefill at 2 cores). K-slicing is
arithmetically right but 4.5x slower below 2048; slicing to 2048 to reach the
lever pays 1.24x.
Also records that "DO NOT RAISE IT" was an artifact of an unpaired knob, that
the shipped split is already maximally weight-biased, and that ORK_I4_DBNK
overshoots H at non-dividing K.
The native-W4A4 prefill gap is program count, not submits, and Wb dominates H 43 to 19
Carry the reset retraction into the Experiment Log entry too
The 4-bit ladder beats i4a8 on quality; the 18x is mode resets, not the tier
Four packs on one model with disjoint GPTQ calibration. Arm T (SmoothQuant +
GPTQ + attn_v/ffn_down promotion) reaches PPL 25.78-26.36 against i4a8's 34.30,
so native 4-bit built with the landed ladder is better, not merely acceptable.
The 18x prefill cost was first attributed to the DT_I8_ROT tier from an average.
That was wrong. Promoting to plain DT_I8 instead changed nothing (9.10 vs 9.08),
and ORK_DEBUG_RESET named the caller: the mixed pack takes 221 ACT_RESETs
(110 each way across I8_CHAIN <-> I4_CHAIN) where both homogeneous packs take 1.
At the 105 ms/reset already measured here, that is 23.1 s of a 25.0 s run.
Also corrects the stale "~54x" native-prefill claim (real 3.62x) and its stated
cause, records that native buys nothing at decode, that GPTQ already skips the
int8 tier, that ORK_GPTQ_CLIP_MIN=0.531 would regress the live default, and that
stub vs full GGUF silently changes which engine runs decode.
Record the mmap weight-ownership bug and the dead ggml-ork correctness gate
The device iface aliased get_host_buffer_type to get_buffer_type, so the
loader's "avoid a host buffer under mmap" rule disowned every weight: 311
tensors to CPU before, 115 after. Compute was never stranded, ownership was.
Also records that tests/test-ork-cache.cpp is red on pristine diverged-history
and cannot distinguish a backend failure from a wrong answer, and what to use
as a gate instead.
Experiment Log: 2026-10-09 (V) the colsplit's fixed per-call cost was cache maintenance over dead bytes
Experiment Log: 2026-10-09 (IV) the domain-reference leak, root-caused
Index the PR #390 result, and flag it on Kernel-Modifications
The 'Build status of #390' section said the series had never been booted. It has
now, on a second board built as armbian plus exactly those ten commits, and it
is not sufficient: orkd_dom_api fails 3/3 until Change 19 (#patchB5) is added.
Flags the reason it was never caught: the runtime testing that section describes
was done on the armbian tree carrying the full board state, which silently
included B5.
Exp-2026-10-09: the fp16 K-slice win is a tie end-to-end, and why
ork_bench on .236, 3 reps per arm, new arm in the disadvantaged first position:
192.98 vs 192.23 prefill, 6.19 vs 6.20 decode. +0.4%/-0.2% -- inside the ~2%
screen and far below this board's arm-position bias band. A tie.
The reason is dispatch, not the fix. ORK_PROFILE reports run_i8 and nothing
else, because ggml-ork routes even an f16 GGUF through the int8 datapath, so
ork_f16_mm_* is never reached. The 2.3-5.0x measured on fp16 shapes in isolation
is real; it just lives in the training path and in any future fp16 routing, not
in inference as currently dispatched. So the 1.0.160 -> 1.0.165 submodule bump is
safe and neutral.
Records the caveat that makes this honest: ork-driver has run_i8/run_i4 profile
counters and no fp16 one, so 'zero fp16 calls' is an absent line rather than a
measurement -- which is why the timing A/B was run instead of resting on it.
Also indexes both of today's pages in the Experiment Log; the training-step page
had been written but never added to the index, and the log is the index of record.
Method note kept for the next person: sudo -E resets HOME to root's, so a bench
script using $HOME/model.gguf under 'sudo -E sh' resolved to /root, the binary
never ran, and the harness printed blank result lines rather than failing.
Exp-2026-10-08: the clip-grid win SURVIVES held-out (57.90 -> 56.53, 2.4%) — the day's one real quality result
Exp-2026-10-08: mixed-radix rotation — 1.7% seen, 0.33% held-out; a seen-text gain is not evidence
Exp-2026-10-08: HELD-OUT eval — real gap is 1.27x; calibration-on-eval flattered us by ~3.6% relative
Exp-2026-10-08: matched-budget promotion — hand prior beats qerr by 3.8%; plus the running-PPL misread
Exp-2026-10-08: activation clip grid is a null — unswept does not imply truncated
Exp-2026-10-08: correct the clip-grid claim — a plateau across 0.594-0.406, not an interior peak
Exp-2026-10-08: the plateau broke — GPTQ clip grid stopped above its optimum, R 52.65 -> 50.42
Exp-2026-10-08: six more ladder arms — alpha=0.5 optimal, back-to-front a null, ladder plateaued at R/T
Exp-2026-10-08: arm RSG = 52.86, a null against R — RHT/GPTQ substitution confirmed a third time
Exp-2026-10-08 (II): int4 doorbell value collision; arm S = 55.11 beats GPTQ alone
Exp-2026-10-08: the fp16 wide-K head hang was a sizing/emit drift, not WEIGHT_BANK starvation
Exp-2026-10-07: final ladder, and the transform-vs-GPTQ substitution finding
Completes the 12-arm ladder: 1737 -> 48.78 against a Q4_K_M target of 37.12 (1.31x at ~5 bpw, 1.55x at
the capacity-honest 4.0 bpw).
The durable finding is not a number, it is that transform refinements and GPTQ are SUBSTITUTES rather
than complements. RHT is -2.6% without GPTQ and +1.9% with it; act-order is +3.6% with it. Both attack
weight-side conditioning and GPTQ's error feedback already does that work. WUSH's published numbers
show the same shape (+2.8 over Hadamard with RTN, +0.7 with GPTQ), which drops it from the main
remaining hope to plausibly zero on a GPTQ stack -- recorded so nobody invests in a data-aware
transform without re-deriving it.
Also records that this screen cannot resolve under ~2% (RHT reads -1.9% at one rung, +1.2% at the
next; PPL is deterministic per pack, so this is sampling error of 16320 tokens, not measurement
noise), and that the ladder table is not a staircase -- J and A are both measured against G.
Open work reframed around the activation half, with the observation that SmoothQuant and SINQ need
identical machinery (a persisted per-input-channel scale applied before the rotation) and differ only
in how that scale is computed.
Exp-2026-10-07 Part II: the W4A4 technique ladder, with act-order as a measured negative
Rung-by-rung decomposition on a plain dense transformer, each technique isolated on one screen:
1737 (no rotation) -> 153.89 (+rotation) -> 113.11 (+act clip) -> 78.03 (+weight clip) -> 75.97 (+RHT)
-> 57.49 (+GPTQ) -> 49.39 (+promotion), against a Q4_K_M target of 37.12.
Rotation is worth 11.3x; everything else is 1.3-1.5x. Any W4A4 number measured without it is measuring
a broken configuration -- which is what Part I did, because ORK_QUANT=4 alone silently selects inflated
int8 with no rotation and no GPTQ registration.
act-order was implemented and is a NEGATIVE (+3.6%): its benefit comes from group boundaries, and
per-row scales have none, while the disruption to GPTQ's propagation order still costs. Recorded with
the reasoning so it is not re-tried.
Promotion: Q4_K_M's own bump list beats our qerr ranking while spending 6 MiB less, because qerr
measures rounding error rather than output impact and its distribution is flat here.
Also records why NF4 cannot work on native W4A4, why grouping costs ~24x, and why W4A8 is 8 bits
resident and so does nothing for the capacity tier.
Exp-2026-10-07: refute the GDN hypothesis — GPTQ registration is broken generally
The page attributed zero-weight GPTQ registration to the model being a Gated DeltaNet. Tested on
qwen3-0.6b-f16, a plain dense transformer: identical failure, zero weights registered, and the same
bogus n_ff=0 fingerprint. Packing works and takes the int4 tier, so weights pack through a path that
is not the native-W4A4 branch carrying the phase-1 hook.
Moving the ladder to a plain transformer therefore does not route around the GPTQ rung being blocked,
which is the practical consequence and was the opposite of what the page implied.
Exp-2026-10-07: the GGUF 4-bit target line, and GPTQ registering zero weights on a GDN model
Measures the target the 4-bit work is actually aiming at. Every prior W4A4 number was quoted against
int8 or against RTN, so 'how far from parity with GGUF 4-bit' had never been a quantity. On one harness,
one corpus, matched window counts: int8 22.92, Q4_K_M 24.66, W4A4 default 83.96.
Also records that GPTQ cannot currently be measured on this model -- phase 1 registers only native-W4A4
weights, max_k comes back 0, and the run writes a plain RTN pack while exiting 0. A GPTQ arm here would
have reported RTN numbers under a GPTQ label.
And the two instrument errors that produced confident wrong conclusions during the session: the rknpu
counters are gated OFF after a reboot, and a grep that matched the ROUTE-STATS header but not its rows
invented a routing bug that did not exist.
Experiment Log: WEIGHT_REUSE gate is now the law, plus the mc == mcap clause the law was missing
The capacity law needed a second clause: 0x1040 derives from mc, so a SHORT
final M-tile gets a different bank split and reads stale bytes even when
capacity passes. K=1024 (mcap 320) at M=512 -> tiles 320+192 -> silent
mismatch; with the clause it declines and is bit-exact.
Also: do not predict the bank counts. 12 - mcap*K/32768 is wrong below K=2048
(K=1024 emits 0xa2 = WEIGHT_BANK 10 / DATA_BANK 2). Read WEIGHT_BANK back from
the emitted word -- a first implementation predicted them and had its readback
guard scoped to the forced path only, so it shipped the mismatch unprotected.
Gating on the law instead of the K==4096 whitelist found a 1.083-1.084x win at
K=3072 N=192 with many full M-tiles, measured against the position bias. But
eligibility is N <~ 192, so it declines on every production shape we hold; the
only eligible MoE tensor is the router. Inert today, correct for whatever
shapes arrive later.
Experiment Log: DATA_REUSE cannot win — its ceiling is the unsegmented baseline, confirmed over 14 shapes
Structural: the unsegmented baseline already fetches the activation once per
M-tile, which is the floor. Segmenting inflates it nsg-fold; reuse returns it to
the floor. So reuse repairs damage segmentation caused and can at best equal
base.
Searched 14 shapes chosen to favour it (small K => large mcap, wide M, narrow
segw). Zero wins; all below 1.000 and approaching it asymptotically from below
as segw widens, best 0.993. Reuse does real work -- recovering 57/74/82% of the
segmentation cost at K=512/1024/2048 -- but repair caps at undamaged.
Bigger models are worse, not better: mcap shrinks as K grows, so large-K gets
the smallest M-tiles and the weakest activation-to-weight ratio.
Notes the unmeasured largest-K asymptote (two runs failed) and that attention
BMM inverts the premise but is a different code path.
Experiment Log: per-task chaining cost is ~9-13us at M=1, which explains every negative reuse result
Sweeping the forced segment width makes task count an independent variable:
+2 tasks costs 24us, +6 costs 77, +10 costs 90, +20 costs 216 -> ~9-13us each,
under-stated since each base carries the arm-position bias. Bounds the regime of
the ~1.3us marginal figure recorded elsewhere rather than contradicting it: at
large M a task's DMA overlaps its neighbour's compute, at M=1 nothing hides it.
Reuse off vs on at MATCHED task count is null (-16.0/+1.7/+6.1/+23.8us, sign
flips, inside a +-25us floor), calibrated by the harness's own null: at segw=672
reuse cannot fire and the arms land 0.7us apart.
Corrects the earlier '608 vs 608' as %.0f rounding over 4 iterations; at 20 it is
570.7 vs 576.8. The bit was firing throughout (90 readbacks with b12=1).