Skip to content

INT4 BCHAIN Operational Notes

Michael Fischer edited this page Oct 13, 2026 · 2 revisions

INT4 BCHAIN — operational notes

Standing reference for ork_i4_run_bchain_db / bch_db_worker (src/npu/i4/run.c, src/npu/i4/chain.c): the int4 W4A4 M>1 prefill doorbell path. This page is the detail; OPS_REGISTRY.md row run_i4_bchain_db is the index entry that points here.

Moved out of that registry row on 2026-10-13, where it had grown to 30,848 characters on one line — 19% of the whole registry file and 5× the next-largest row. Nothing here is new; it is the same text, sectioned. The dated experiment pages remain the primary record for the 2026-10 work and are linked per section.

Default for M>=2, Sk==1, Sn==1, N%64==0 — otherwise the per-row ork_i4_dyn_begin_mc. Supersedes STRATEGY-A/C, blocking BCHAIN, and CBATCH (removed #51/#52/#53).


1. orki_i4_hcap is a MEASURED TABLE, not a formula

25 probed K (32..8192). No closed form fits them and interpolation is unsafe in BOTH directions. Below K=256 the envelope goes FLAT (128 for K<=128, 64 for 128<K<256) with no relation to 16384/K, which wants 512 at K=32. ceil(16384/K) overshoots at K=3072/3584/5120 and undershoots at 6144; K=768 and K=3072 share fractional part .33 yet round opposite, 22 vs 4.

Unmeasured K fall back to floor(12288/K) — an empirical 0.75× margin, not a guarantee.

★ An overshoot does not merely miscompute. At K=3072/3584 the submit is accepted on every core (submit_rc=0) and the NPU never starts it (hw_elapse=0), so the recover loop resets 6× and fails after ~15 s. This was the "native W4A4 hangs at prefill" report (fixed 2026-08-21).

Rules for adding or changing an entry

  • Grade a candidate H by CHECKSUM only. rc misses the silent-miscompute mode and so does timing.
  • Validate BOTH standalone AND as a slice remainder (K=8192+rem). Standalone is the weaker test — a single agreeing run would have passed the bad K=2560 entry below.
  • H is only ever LOWERED. Raising it past orki_i4_hcap() miscomputes SILENTLY, which is why the global ORK_I4_H knob must never be used as a blanket fix: ORK_I4_H=4 makes all 16 i4 shapes read maxerr=0 while i4-wideK (K=10240) and i4-rem2560 (K=18944) return rc=0 with the wrong golden.
  • ★ Read the table as "the .236 measurement", never as hardware — see §4.

2. The K=2560 slice-remainder non-determinism (fixed 2026-08-22)

K=2560 agreed six ways STANDALONE at H=7 and is correct there, but as a slice remainder H=7 varied run-to-run on ~40–60% of runs. K=18944 — the real 7B ffn_down — is 2*8192 + 2560.

Per-sub-tile checksums isolated it to the 2560 tile alone (the 8192 tiles were bit-identical every run). Not the accumulate, not recover, not an early poll, not an overrun. The rate falls smoothly with H (7: ~50%, 6: 5/8, 5: 4/8, 4: 0/8) ⇒ stale-CBUF residue exposed by a marginal H. K=2560 is now 4, where 2816..4096 already sit. All 25 entries were re-confirmed as remainders, 6 runs each.

H=7 was never a win regardless: 2.1 s/run against 1.0 ms at H=4.

⚠ An earlier "kernel timeout + non-idempotent retry" claim was WRONG — read from one sample. Ruled out by measurement: our recover loop (MC-RECOVER=0), submit rejection (0), extra soft resets (+1 only), the submit itself (dyn_end done=3/3 in 496 µs), our submit timeout (wall flat across 500..4000 ms), an early poll (2 ms dwell no help), an overrun (padding part no help).

★ The underlying defect was in the SHARED drain, not int4

ork_dyn_progress returned the HIGHEST landed index, not a count, and ork_dyn_end tested last >= h->S-1 — so a HOLE at a non-final task read as success and de-tiled partial output ([dyn_end] done=2/3 last=2 ... INCOMPLETE on the corrupt runs), while the recover loop and the auto dump both sat in the same else and never fired. The recover loop was never broken, just UNREACHABLE behind the faulty test.

Fixed in ork_dyn_progress (stop at the first hole — every caller reads it as a FRONTIER: dump names prog+1, ork_dyn_remaining uses P-(p+1), queue halts at prog+1+HEADROOM). With the K=2560 H=7 bug reintroduced, test_slice_rescue PASSES 3/3 with [MC-RECOVER] reset + resubmit where it failed 3/3 before.

Regression: test_slice_rescue case i4-rem2560 (K=18944, 5 runs, all must be bit-identical — a single golden compare passes ~50% of the time with the bug, so the repetition IS the test).

Validated at default + ORK_SSM_KEEPWARM=0 + ORK_MIXED_NOTHRASH=1, reset count 51 == baseline 51. (mode_probe was unusable as a gate at the time — hangs at BASELINE, rknpu_gem_fault.)

3. The envelope has a DRAM-CLOCK term — and underclocking is not a remedy

orki_i4_hcap() and Wb were measured on .236 (LPDDR4X 2112 MHz) and do not hold on .235 (LPDDR5 2400 MHz), where four i4 shapes and test_slice_rescue produce the overshoot signature (submit_rc=0, hw_elapse=0, int_status=0x0, rc=-1).

Measured cause: the DRAM clock. Not the silicon (same chipset, and the two boards agree on the timing physics at every geometry both can run) and not the NPU clock. Capping NPU devfreq 1000→800→600→400 MHz leaves the bad shape failing at every step; dropping DMC 2400→1968 makes it PASS with the correct checksum (f28a3df8f9118d1d), and 1320 likewise. Control: holding the governor at userspace and varying only frequency gives 2400 FAIL / 1968 PASS / 2400 FAIL. So it is a DMA/timing envelope, not a CBUF capacity one — a capacity limit cannot care how fast DRAM runs.

The threshold is SHAPE-dependent, and 1968 is not low enough for production shapes

Re-measured 2026-10-12 on .235, with cur_freq read back at every point AND sampled during every call, NPU devfreq held at 1000 MHz throughout:

  • Discriminator K=2048 N=64 M=16: shipped H=8 FAILS at a verified 2400, PASSES at a verified 1968 and 1320. Its threshold really is in (1968, 2400].
  • Production shape K=2048 N=2048 M=128, full-random: H=5/6/8 all FAIL at a verified 2400 and at a verified 1968 (rc=-1, ~16.5 s, checksums differing run to run), all PASS at a verified 1320, H=8 bit-identical to the H=2 anchor on 10/10.

That reconciles two earlier readings: the pack build pinned at 1968 kept missing because 1968 is still ABOVE the production-shape threshold, not because the clock is irrelevant — so the earlier "the clock does not govern the envelope at production shapes" is withdrawn in turn.

★ Underclocking is NOT a remedy — ⚠ RETIRED, see the 2026-10-13 correction below; this no longer reproduces: 1320+H=8 runs that shape in 4183 µs against 2400+H=4 at 3503 µs, so the DRAM de-rate costs ~19% more than the larger H wins.

★ And a board cannot be classified by cur_freq alone. .236 is LPDDR4X at 2112 (its OPP maximum — it cannot be raised) and takes H=8, while .235 is LPDDR5 and fails at 1968. Memory type, not the integer clock, is what the envelope tracks. The OPP tables do not overlap (.236: 528/1068/1560/2112; .235: 534/1320/1968/2400), so a matched-clock experiment between the two boards does not exist.

tools/i4_clk_envelope.sh re-runs the discriminator in a couple of minutes on any new part. This was a sampling limit (one memory configuration), not a modelling error.

★★ CORRECTION 2026-10-13 (later the same day): the clock term is a STEP, not a slope — and the 19% above is RETIRED

Re-measured on .235 (6.1.115-vendor-rk35xx-fix) against a clean clone of origin/main 51cc4d5, i4_ladder_probe (absolute CPU reference), ORK_I4_NOLADDER=1, full-random, 6 reps, BOTH arm orders, DMC pinned userspace and sampled every 0.4 s during every arm (every arm min == max == set). 24 arms, 0 faults.

(a) The envelope is FLAT from 1968 to 2400. At every K tested the set of working geometries is identical at 2400 and at 1968; it opens only at 1320. A −18% de-rate buys zero envelope width and costs 10–16% in time. So the governing variable has a threshold in (1320, 1968] and does nothing above it. "DRAM clock governs dispatch" is too strong as stated — it governs it at a step.

(b) ★ The ~19% underclocking penalty DOES NOT REPRODUCE on the current tree. Retire the figure. Same cell, K=2048 N=2048 M=128:

then now
DMC 1320 + H=8 4183 µs 3326–3366 µs
DMC 2400 + H=4 3503 µs 3397–3445 µs
verdict de-rate costs 19% de-rate is 1–3.5% FASTER

The Wb/ladder and completion-gate work closed the gap entirely. Decomposition: the de-rate costs 15.6% at matched geometry (3397 → 3929 at 64:4) and H=8 hands 14.3% of it back (3929 → 3366).

(c) ★ The boundary is GRADED, and "PASSES at 1968" above is really a RECOVERED-pass. At the discriminator K=2048 N=64 M=16, fixed geometry 64:8, as DMC falls: 15.64 s (fail) → 110 ms (passes, ~1600× inflated = an internal self-heal) → 66 µs (clean at 1320). A checksum-only gate calls 1968 a pass. This is exactly the "checksum-only grading ships self-heals" trap in §9 catching the author of §3.

(d) ⚠⚠ A SILENT MISCOMPUTE INSIDE THE ENVELOPE — new, unexplained, and the most important line on this page. K=2048 N=2048 M=128, DMC 1968, descending order only, cell 64:4 — a geometry that is 6/6 clean at 2400 in BOTH orders — returned rc=0 with 198,583 wrong elements on 2 of 6 reps, ~2.00 s each. It did not fail; it answered wrongly. It occurred only when run after the failing H=5/6/8 cells, so a REFUSED dispatch appears to leave state that corrupts the next, legal run. Same family as the CBUF-residue mode §6 documents (which is why the ladder resets before a retry), but here it crosses into a different cell and is not covered by that reset. Unchased. ★ This is the strongest argument in the dataset for running both arm orders on everything.

(e) 2736 is probably NOT the lever, and a better one is in the same data. Since the envelope is already at its floor across 1968–2400, there is no mechanism left for 2736 to narrow it — expected outcome is same envelope, more bandwidth, worth maybe 3–5% at K=2048 (inference from the flat region, not a measurement). Against that: K=1024 at DMC 1320 with Wb=320 ran 1309–1468 µs against 1980 µs at 2400 — 1.35–1.5× — if its 3–4/6 reliability can be fixed. That needs no bootloader.

The 2736 feasibility verdict: not DTB-only, and not a Rockchip RK3588 blob at all

  • The DTB does not choose the frequencies; the DDR firmware does. rockchip_dmcfreq_adjust_opp_table() (vendor drivers/devfreq/rockchip_dmc.c) takes freq_info_rate[] from the SMC ROCKCHIP_SIP_CONFIG_DRAM_GET_FREQ_INFO and overwrites DT OPP i's rate with it, taking the voltage from the first DT OPP whose rate is ≥ that. Confirmed by prediction: DT carries 528/1068/1560/2750 yet the live table reads 534/1320/1968/2400, and the predicted voltage mapping matched 4/4 (534→725 mV, 1320→800, 1968→875, 2400→875). The DTB can only LOWER the ceiling, never raise it — and it already contains opp-2750000000 @ 0.875 V, so no DTB edit would be needed for 2736 anyway.
  • ★ Rockchip ships no RK3588 DDR blob above LP5 2400, at any version. rkbin bin/rk35 has only rk3588_ddr_lp4_1848MHz_lp5_2112MHz_v1.24 and rk3588_ddr_lp4_2112MHz_lp5_2400MHz_v1.24. The widely-cited "2736" blob is an RK3576/RK3572 part (rk3576_ddr_lp4_2112MHz_lp5_2736MHz_v1.13). .235 is already on Rockchip's newest and highest RK3588 firmware (v1.24) and its LP5 ceiling is exactly the 2400 observed.
  • Not conclusively closed: rkbin/tools/ddrbin_tool_user_guide.txt gives RK3588's tool-settable LP5 range as [400 MHz – 2750 MHz], and rk3588s.dtsi defines a 2750 MHz OPP, so ddrbin_tool.py can encode 2736. ⚠ Encodable is not validated — Rockchip shipping 2736 for RK3576 and not for RK3588 is evidence they did not qualify it here.
  • The binding gate is the DDR blob inside the idbloader, and the RKNS v2 header carries a SHA256 per component (all 5 verified matching), so a byte splice breaks the hash — the loader must be rebuilt with mkimage.

★ A better experiment the same capability unlocks. ddrbin_tool can set f1/f2/f3 as well as f0, on BOTH boards (.236 LP4x is settable over [306.5, 2133]). So the matched-clock cross-board experiment this page calls impossible — "the OPP tables do not overlap, so a matched-clock experiment between the two boards does not exist" — is constructible: give one board an FSP point at one of the other's frequencies and memory TYPE becomes the only remaining difference. Given (a), that is worth more than a fourth point at 2736.

Board forensics worth keeping (.235, 2026-10-13)

  • Boots from microSD (/sys/block/mmcblk1/device/type = SD; host mmc1 = fe2c0000.mmc; no mmcblk0). Two complete loaders exist: SPI carries DDR v1.24 + bl31-v1.56, U-Boot built Oct 09 2026 (hand-flashed, NOT the Armbian SPI package); the SD carries DDR v1.20, U-Boot Aug 19 2026, and is byte-identical to the installed Armbian package /usr/lib/linux-u-boot-vendor-rock-5b-plus/u-boot-rockchip.bin — so the SD loader is reproducible from a package already on the board.
  • ⚠ WHICH LOADER TRAINS THE DRAM IS STILL OPEN. The only distinguishing runtime artefact is the U-Boot build date, which the kernel does not expose. Patching the losing carrier yields a null that looks exactly like "2736 unsupported." Settle it first, two ways: a serial console (wired and reachable — console=ttyS2,1500000, serial-getty@ttyFIQ0 active — the DDR blob prints its fwver and trained speed at boot), or a reboot plus rk3588_dmc_init()'s dmc: current ATF version 0x%lx, which should differ between SPI's tagged BL31 and the SD's.
  • Boot config is Armbian boot.scr/armbianEnv.txt with user_overlays=rk3588-a76-2400 rk3588-wdt, and boot.cmd restores the base DT if any overlay fails to apply — so a DTB-level change here is self-healing and revertible from a running system. (Moot for 2736, per above.)
  • Verified backups: full 16 MB SPI (e2b51f6a…) and SD first 16 MB (39ef3698…), plus DTB, overlays and boot config, sha256-checked on both ends, at ~/ork-outbox/boot-backup-2026-10-13/ and /Users/michael/ork-board-evidence/235-boot-backup-2026-10-13/.

4. It is NOT a single-axis H ceiling — and the binding axis is often Wb

2026-10-12, .235 at a verified 2400, N=2048 M=128, full-random, each arm in its own process, shipped geometry FIRST so the result is not an order artifact:

K shipped Wb behaviour
2048 64 behaves like a ceiling — H<=4 passes, H>=5 fails
1024 128 EVERY H fails (16, 8, 4, 2) — yet every one of 16/8/4 passes at Wb=64
512 256 the direction INVERTS: H=32 and H=16 pass at ~1.33 ms while H=8 and H=2 fail

So "lower H is safer" is false, the binding constraint at K<=1024 is Wb and not H, and the ladder's Wb=64 step — not its H=4 step — is what rescues those shapes. Do not fit a closed form to orki_i4_hcap from one board's data; there is no monotone ceiling to fit.

⚠ Wb is also a modelling gap: it is documented as architectural bank containment, which has no M term, yet Wb=256 fails at M=8 and works at M=64 at the same K and N.

Two independent walls had been conflated into one

60-point (K,Wb,H) map via i4_storm_probe, ORK_I4_NOLADDER=1, every arm checksum-graded:

  • (a) Activation/H wall — violating H <= DBNK*16384/K silently miscomputes at normal speed (K=1024 H=32 and K=512 H=64 at DBNK=1; both become CORRECT at DBNK=2, confirming the i4_bank_sweep form).
  • (b) Weight-tile/dispatch wall — the submit is accepted and the output never lands, governed by Wb. Wb=64 — a SINGLE 64-column output block per program — never missed, 26/26 points from K=512 to K=8192. At Wb>64 it tracks K*Wb: <=65536 int4 elements (32 KB, one CBUF bank) always dispatched; =131072 (two banks) is the erratic regime; >=262144 missed 10/10.

★ So it is not weight capacity — K=8192 at Wb=64 is K*Wb=524288 and runs fine — it is specific to MULTI-N-BLOCK programs, whose output surface is 2D (block stride 4*Hg*64). The shipped Wb = (131072/K)&~63 nevertheless aims at the TWO-bank value — the comment calling 131072 elements "one CBUF weight bank" is off by 2× — so every K in 1024..2048 starts in the erratic regime.

At Wb=64 every K reaches its full activation budget H*K=16384 except K=2048, which stops at H=4; that hole is the whole K-row, reproducing at N=512/1024/2048/4096 and M=64/128/256. Nothing moves it: ORK_I4_WREUSE=0, ORK_NPU_MC=1/2, ORK_MOE_SRAM_SCRATCH=1 (the whole per-program working set on the second port — the prior "SRAM is NULL for NPU reads" finding was a bandwidth measurement and did not cover this, so it was retested), ORK_I4_DBNK=2, ORK_NO_PINGPONG=1, and the burst sweep below were each measured at K=2048 H=8 and each left the miss unchanged.

⚠ ORK_I4_1010 is worse than useless here: unlike its int8 twin it is NOT an inert perf hint on the int4 path and corrupts the known-good H=4 as well.

★ The failure is not "the job never started": the ORK_MC_DIAG census reports ~36% of the WRITTEN cells still at the seed, scattered over hundreds of runs spanning the whole surface, and the fraction RISES as cores are removed (nc=3 36%, nc=2 43%, nc=1 49%). Individual programs inside an accepted chain drop — a race, not a capacity law.

5. DMA burst length is not the mechanism, and the sign is the opposite of the guess

RK_CNA_DMA_CON0 (0x201:0x1078) is not absent from the int4 path — REGCMD_I4 ships it packed as 0x000f1078, 0x0201000f, i.e. 0x000f000f: both WEIGHT_BURST_LEN and DATA_BURST_LEN at maximum, the same value int8 uses.

Swept down via ORK_I4_BURST (src/npu/i4/run.c) at the storming shipped geometry: no value opens H=8 (0x8/0x4/0x2/0x1/0x0 and the mixed 0xf0001/0x1000f all fail, several with the ENTIRE surface untouched, sat+=262144), and worse, 0x00080008 and 0x00010001 BREAK the known-good H=4 that passes at 0x000f000f.

The knob plainly reaches the hardware; shortening the burst is simply harmful. The maximum burst is required — do not lower it.

6. The geometry LADDER is the fix that holds

It responds to an actual miss rather than predicting one. On a doorbell miss, retry with a narrower geometry, each retry preceded by ork_npu_mode_reset(). Every step is bit-exact, and a step that changes nothing is skipped, so a board under the threshold pays nothing.

.235 with NO env knobs: i4 4×rc=-1 → all 16 maxerr=0; test_slice_rescue FAIL → PASS with every golden matching the .236-generated value.

★ The reset is load-bearing. Without it i4-refuse returned a WRONG answer (308,202 mismatches) instead of a miss — a geometry the part will not take leaves CBUF residue, the same mode §2 documents.

ORK_I4_NOLADDER=1 for an A/B.

Rung 1 must be TODAY'S SHIPPED WIDTH (2026-10-13)

The ladder used to go wbdef -> Wb=64 -> Wb=64,H=4, so a part refusing a widened rung 0 forfeited the Wb it DOES take and landed two rungs down, permanently for the process. Rungs are now orki_i4_wbdef(K,N) -> (131072/K)&~63 -> 64 -> 64,H=4, with "a rung that is not a DIFFERENT geometry is skipped" generalised over the four.

orki_i4_wbdef is min(320,N) at K<=1024 with hcap>=12, else the shipped floor; ORK_I4_NOWIDE=1 pins it to the floor for an A/B. Measured (i4_ladder_probe, absolute CPU reference, 6 reps, both arm orders): every arm 6/6 bit-exact at every K, and the predicted −25% at K=768 does not occur — rung 0 Wb=320 is refused once and every call then runs Wb=128 H=22, today's width, at 1573 µs against the shipped 1559 (rung 1's absence would have put it at 1917). Worst deviation anywhere is +1.6% at K=1024, where both arms dispatch the same geometry and the A/A spread is 1.3%. K=512 is the only K where this part takes the wide tile, −5.3%, which that K's 5.6% A/A spread swallows — direction favourable, not established.

The rule's cost is the LEARN: K=768 now pays 1.32 s once per (K,N) per process where it paid none; K=1024 pays 2.54 s instead of 1.33 s.

★ The ladder was re-paying the refused rung on EVERY call

Nothing recorded that the part had already refused a geometry — measured K=2048 N=2048 M=128 at 15.86 SECONDS per call against 3.59 ms for the rung it lands on, and K=1024 at 15.86 s against 2.23 ms.

Fixed: a per-part (K,N) -> rung memo that only ever TIGHTENS, plus ladder_probe on ork_dyn_chain so an attempt with a bit-exact fallback skips the six recover rounds and caps its wait. Measured: 15860 → 3435 µs (4.6×) at K=2048 and 15857 → 2141 µs (7.4×) at K=1024, same checksums, with a one-time ~1.33 s per (K,N) to learn the rung; K=512 (which never missed) unchanged at ~1.25 ms.

Two traps found while fixing it:

  1. the reset before a retry must be conditioned on having missed, not on "we are past rung 0" — resetting on a memoized start costs a full ACT_RESET + cold re-warm, 216 ms per call, which ate 98% of the win;
  2. a rung that is not a DIFFERENT geometry from the one above it must be skipped, or the memo records a rung the shape can never reach.

★ A ladder PROBE must size its wait from est too. recov_max=0 routed it past the learned window onto a hard-coded 1 s while the EWMA for that very geometry read 22.6 ms — each probe over-waited 7.3× (1000 → 136 ms measured, prefill 76.4 → 77.1 tok/s).

The shipped geometry ALREADY failed on .235 at two production K

Same method, ORK_I4_NOLADDER=1 ORK_I4_NOTAIL=1: K=1024 Wb=128 H=16 (SHIPPED) is 1/6 ascending and 0/6 descending; K=2048 Wb=64 H=8 (SHIPPED) is 0/6 in both orders, misses ~15.6 s each. Both fall to a rung that is 6/6 (Wb=64 H=16 at 2161–2233 µs; Wb=64 H=4 at 3499–3629 µs). So every K=1024/2048 int4 BCHAIN call on this part runs one rung down after a ~1.33 s learn — and has since the ladder landed.

orki_i4_hcap's K=2048 entry (8) is not this part's ceiling (4 is), but its K=1024 entry (16) is not at fault either — H=16 is 6/6 at Wb=64, so the failing axis there is Wb. ★ Neither is a table error to patch: both are DRAM-configuration-dependent DISPATCH races, so a second hcap table keyed on the board would encode a race as a capacity law.

(The one K=1024 Wb=128 rep that landed ran 1663 µs against 2161 for the rung it falls to — the shipped width is the FAST geometry at that K; this part simply cannot dispatch it.)

7. Tail-group balance — the cost is the LAST M-group, not the program count

2026-10-12, .236, i4_e2_probe (absolute CPU reference), 6 reps, every sweep ascending AND descending, full-random.

H = orki_i4_hcap(K) is a LOCAL WORST wherever hcap divides M: K=512 N=2048 M=128 Wb=320 measures 1326 µs at H=32 against 1147 at H=31, bit-identical, although H=31 raises the program count (NG 4→5).

The governing variable is neither H | M nor a power-of-two H — both REFUTED by discriminating cells. At M=130 (nothing divides) every H in 26..32 gives the same NG=5 and time still falls monotonically 1280 → 1173 as H rises; at K=768 (hcap=22, not a power of two) M=132 measures H=22 1345 / H=21 1237 / H=20 1286 / H=19 1353 / H=18 1282 — H=19 does not divide 132 and is slow, H=18 does not divide it and is fast.

★ The variable is r = M-(NG-1)*H, the row count of the FINAL M-group, at ~6 µs/row at Wb=320 on top of the ~6 µs/row every row costs. H | M is simply the maximal-r case (r == H).

Two independent confirmations: adding ONE row at fixed H (M=128→129, H=32, +0.8% work) runs 21% FASTER (1400 → 1157 µs) because r drops 32→1; and the effect DISAPPEARS entirely under ORK_I4_BLOCKING=1 (M=128 803 µs vs M=129 827 µs — work order restored), which removes the SENT16 seed and the drain-verify poll. So the penalty is host-side, in the completion poll that gates on the chain's LAST program and scans its r*Wmax*64 output cells. It is not NPU time.

That predicts, and measurement confirms, scaling with the N-tile width: at K=512 M=128 the H=32→31 gain is 0% at Wb=64, 0–2% at 128, 12% at 256, 14–17% at 320.

gam was 1.8× light — recalibrated by FIT, on two boards' data

The gate was Wb>=256 because .236 measured null at 128. .235 then measured it real at Wb=128 (K=768 hcap=22 → H=21: 2.7%; K=512 hcap=32 → H=31: 3.3%; end-to-end 1536 → 1494 µs, 2.8%), and .236 independently measured the same sign (−2.1% at M=64, −4.5% at M=96; its other M are A/A because the search does not move there). So the gate moved to Wb>=128 on two boards' data, not one.

gam was then FITTED rather than assumed: T(h) ~ alpha + A*g + B*r at fixed M/K/N/Wb over 9 (K,Wb) grids (K=256/512/768, Wb=64..512, 13 h cells spanning g=4..8 and r=2..32, M=128). B — the per-tail-row cost, which is what gam prices — is 1.21 / 3.35–4.64 / 5.96 / 7.12–7.19 / 7.09 / 13.02 µs at Wmax=1/2/3/4/5/8, i.e. ~1.7*Wmax µs/row — exactly the r*Wmax*64-cell scan the mechanism predicts.

In the model's 3-per-program units that is gam = 3*B*tcrit/A = 2.68 / 4.60 / 3.01 / 4.04 / 7.73 / 7.46 on the six well-conditioned grids (through-origin LSQ 1.82*Wmax; the three ill-conditioned grids all sit ABOVE it, so a linear rule is the conservative read). So Wmax+Wmax/4 was 1.8× light and is now 2*Wmax.

Verified against the MEASURED argmin over an M sweep (M=64..256 at K=768 Wb=128, K=512 Wb=128 and Wb=256; 18 grids, window h in [hcap-3, hcap]): 2*Wmax picks the measured optimum in 18/18; the shipped 1.25*Wmax in 16/18 (it loses K=768 M=144, where H 22→20 measures −1.2%, and it has no gate at all at Wb=128, where the window optimum is worth −3.4% to −13.7% at K=512 and −2.0% to −5.0% at K=768). 3*Wmax picks identically to 2*Wmax on all 18, so the smaller — and nearer .236's slower-DRAM regime — is taken.

Earlier production A/B at the shipped Wb=256 (K=512 N=2048, both orders, bit-exact and checksum-identical at every point): M=128 1340 → 1179 (1.137×), M=192 1883 → 1751 (1.075×), M=256 2528 → 2343 (1.079×). Controls unchanged. ORK_I4_NOTAIL=1 reverts.

★ And a new guard: r >= 2

The search minimises r, and r == 1 — a final M-group of ONE row — is a DISPATCH HOLE on .235: K=768 Wb=128 is 0/9 across three independent (M,h) with r=1 (M=43/64 at h=21, M=96 at h=19; ~15.6 s doorbell miss each) while r=2 at the SAME h is 3/3; at K=512 r=1 dispatches but is never faster than its r>=2 neighbour. Without the guard the recalibrated model steers K=768 M=64 and M=96 straight into it.

The hole is not created by this change — the earlier Wb>=256 gate already picked r=1 at e.g. K=512 M=63 — and it is still reachable when the heuristic does not fire (natural H=hcap with M%hcap==1). That residue is unfixed and belongs to orki_i4_hcap, not here.

8. The completion gate — three corrections in sequence

Full record: Exp-2026-10-12 Shared Doorbell Drain Defects, Exp-2026-10-13 INT4 BCHAIN Completion Gate.

  1. Three defects in the SHARED doorbell spine (not int4 bugs, merely exposed by a workload that misses constantly): the miss-window EWMA key omitted the geometry (orki_opkey now takes orki_geomkey(H, Wb)); the 50 ms miss floor sat below one poll pass and is now measured as 3× the first complete pass; and the recover re-seeded all of nout including the 75% of each slot that is padding, after which the adjudicator could never accept again. Related: collide_ok no longer LATCHES for the chain's life.
  2. The collision verdict has to reach the FRONTIER, not just edone. ork_dyn_progress re-polls through ork_dyn_done_i, so clearing collide_ok before that call discards an accept and every W4A4 model decode dies with progress=-1 (regression 99771bc, fixed 2026-10-13; guarded by the sattest case).
  3. ★ And once that verdict counts, it has to be a CENSUS. Mode 4's "a landed program has >=1 non-sentinel cell" cannot see a program caught MID-WRITE, so on .235 it accepted surfaces 25–50% unwritten and i4/test_slice_rescue returned seed cells as data (maxerr ~32767). The gate now refuses any entirely-seed 64-cell block — every observed tear is block-granular — and backstops with a 0.1%-of-written cap, while a real saturated-accumulator collision (1 cell, 0 dead blocks) still passes.

9. Probes, and how to grade

probe use
i4_ladder_probe (tools/re/i4_ladder_probe.c) ladder / geometry changes. Grades absolutely
i4_e2_probe absolute CPU int4×int4 reference; tail-group and cost-model work
i4_storm_probe storm detection: repeated, timed, per-call checksum + sentinel census, so self-heal / silent miscompute / value collision are distinguishable
i4_hcap_probe ⚠ cannot tell those three apart, and uses a PATTERNED generator
tools/i4_clk_envelope.sh re-runs the DMC discriminator on a new part

⚠ i4_storm_probe grades against its own iteration 0, and that anchor is itself a geometry — a consistently-wrong config agrees with itself. Adequate for storm detection; not for grading a change that alters the geometry. ★ A ladder A/B changes the geometry, and every other anchor in this op is a geometry, which is why i4_ladder_probe grades against a CPU reference.

Implementation notes: parallel build + de-tile (pool); seeds/polls the WRITTEN strided int16 cell-set per cache line (per-element dc was a ~60 ms host wall; per-program validate_regcmd is gated to ORK_I4_VALIDATE, which was the wide-N tax); recovery is serialized on the caller after join (a global RESET cannot run with live cores).


Related pages

  • Exp-2026-10-12 INT4-BCHAIN Dispatch Envelope — the primary record, §§1–27
  • Exp-2026-10-13 INT4 BCHAIN Completion Gate — the gate, the fence negative, and §7 on the board-dependent premise
  • Exp-2026-10-12 Shared Doorbell Drain Defects
  • Exp-2026-10-09 The Domain Reference Leak

Clone this wiki locally