VINDEX3: executable container, and a refused expert that cannot become a token - #197
Open
chrishayuk wants to merge 45 commits into
Open
VINDEX3: executable container, and a refused expert that cannot become a token#197chrishayuk wants to merge 45 commits into
chrishayuk wants to merge 45 commits into
Conversation
parse_layer_weights_header read the magic, skipped the version field, then parsed the offset table as num_entries x 32 bytes unconditionally. A file whose entry stride differs from 32 would not bounds-fail: it yields offsets that are still inside the file, so get_layer_entry_bytes hands back a plausible byte range from the wrong expert and the model produces plausible wrong numbers. Same class as the XSTRIDE and BufferCache::get_bytes bugs — a wrong stride gives a plausible number from the wrong place, not a crash. The single production caller (format/weights/load/q4k.rs) already treats None as "skip this layer", so an unreadable file degrades to a clean miss rather than a panic. This is cheap only until the first v2 file exists anywhere, including in a fixture. LYRW v2 files land in the following commits, so it goes first and alone.
… programme
VINDEX2 is the successor serving container for sparse models — one
extraction, then vary what is loaded, where it resides, what precision it
uses, and whether a component is executed or queried, without rebuilding
the index.
Five new pieces over VINDEX1: per-region quantisation, multiple physical
segments per logical bank, routed/shared/dense bank kinds, a validated MoE
programme manifest, and representation variants that profiles *select*
rather than request.
Spec is at draft-2. Draft-1 was unimplementable as written; building the
reader found three binary-layout gaps, now folded in and marked inline so
the amendment trail survives freeze review:
§6.2 bank descriptor gains region_schema_count (without it the schema
table has no bank boundaries) and flags carrying the §15.2 browse
mode; record is 24 B, with the two u16 counts ahead of the u32 dims
§6.4 schema record padded 18 -> 20 B with an explicit reserved u16, so
the two u32 dims stay 4-byte aligned
§6.3 per-segment entry-table coverage is now normative
§6.5 unknown role/format/packing tags are preserved at read time and
refused at capability-check time — the mechanical form of the
additive-extension precedent
The experiments document pre-registers five gates (V2-0..V2-4), nine
experiments (E0..E8) and seventeen falsifiable priors, with decision rules
fixed before any arm runs. Conformance envelope is Gemma MoE -> GPT-OSS ->
Kimi-Linear-48B -> Inkling-Small -> K3; K3 is extracted once, last, into
the frozen ABI.
E0's corpus is amended from draft-1's "existing production v1 indexes",
which named nothing on the rig — the experiment would have passed by having
no subject. It is now constructed and pinned: C1 a fresh Gemma v1 extract,
C2 pulled hub artifacts, C3 golden outputs captured from the pre-v2 binary
and committed. C3 is load-bearing: without pinned goldens, "zero
behavioural regression" degrades into "both binaries agree", which a shared
bug satisfies.
Also carries docs/lyrw-v2.md, the narrower predecessor programme whose
banked dec results constrain three VINDEX2 arms (fixture D superblock
alignment, E4's absent K3 fidelity axis, the streaming-writer blocker).
Registry programme: vindex2 (14 entries).
The storage half of V2-0. LYRW v2 describes storage only: banks, entries,
region schemas, offsets and formats. It carries no programme identity — the
MoE manifest binds bank_id -> programme, and keeping that in exactly one
place is why "binary says programme 4, manifest says gpt-oss-expert-v1"
cannot happen.
Layout: header, bank descriptors, segment descriptors, per-bank region
schemas, per-segment entry tables, then 64-B-aligned payload regions.
The writer streams. The v1 writer took &[LayerEntry] — every expert's
quantised bytes, fully materialised — which at a K3 routed layer is 39.4 GB
in, ~118 GB f32 intermediate and 24.3 GB out against 128 GB of RAM. That is
a property of the API signature, not the machine. This one emits the
tables, reserves the entry table, appends one region at a time and
backpatches on close; retained state is one (offset, length) pair per
region and never the payload.
The reader resolves (bank, entry, role) to a byte range without copying, so
untouched up/down pages cost nothing under mmap.
Refusals are the point. The failure this format defends against is not a
crash — it is a table read at the wrong stride landing on offsets still
inside the file, returning plausible bytes from the wrong expert. So every
check fails closed and names what was wrong: a v1 file is refused by
generation ("requires the VINDEX1 loader", never a parse error), a foreign
file by magic, a short file by which table was truncated, an out-of-range
region by bank, entry, role and the file length it was checked against.
Unknown role/format/packing tags are preserved rather than rejected —
refusal belongs at capability-check time, or a browse-only reader chokes on
a down region it never touches (spec §6.5).
Bank descriptor field order is pinned byte-by-byte against spec §6.2, since
a writer/reader divergence there produces wrong bytes rather than an error.
20 files, 151 tests, no file over 260 lines. Per-file line coverage: 13 at
100%, lowest 98.9%.
Not yet built: MoE manifest schema, profile inheritance and variant
selection, derived authority as weakest-link, programme-traversal capability
checking. All of them sit on this container.
index.json's `version` becomes the sole generation discriminator, enforced rather than documented. `load_vindex_config` previously read index.json and never checked it, so a VINDEX3 directory deserialised into VindexConfig on the strength of its shared field names and the VINDEX2 loader proceeded against a layout whose weights live somewhere else — a served model with wrong weights, not an error. Both generations are renamed so the number matches the discriminator: generation index.json.version LYRW format_version VINDEX2 2 1 VINDEX3 3 2 The previous VINDEX1/VINDEX2 naming put a permanent off-by-one between the name and the field every loader dispatches on. LYRW keeps its own sequence and trails the container by one, permanently — the two are different artifacts with different lifetimes, and LYRW's numbering was already right. That offset was load-bearing and nearly shipped wrong: Lyrw2Error's generation mapping was identity, correct only under the old naming. A LYRW v3 file would have named "VINDEX3" when it means VINDEX4. Two tests caught it; the mapping is now explicit and pinned across a range. Unknown role/format/packing tags are preserved at read time and refused at capability-check time (spec §6.5) — the mechanical form of the additive- extension precedent, and what stops a browse-only reader choking on a down region it never touches.
Extracting a MoE model whose experts are stored as separate tensors per
expert (`experts.{id}.w1/w2/w3` — OLMoE, Mixtral, DeepSeek) produced an
index that reported success, passed `larql verify` on every file, sliced
cleanly on all eight presets, walked cleanly, and then panicked on the
first decoded token:
pure-MoE layer 0 has no expert weights: the vindex is missing its
per-layer expert store (layers/layer_00.weights)
write_per_layer_moe_kquant is gated on `is_moe() && expert_format ==
PackedBF16`. The default expert_format is PerExpert and only Gemma 4
overrides it, so the entire separate-tensor family returned early and
wrote nothing. The gate had already been widened once, from
`is_hybrid_moe()`, with a comment naming this exact failure mode — the
widening did not reach far enough.
Found by extracting OLMoE-1B-7B while building the E0 preservation
corpus, which is what that corpus is for.
The new writer reuses `quantize_dense_entry`: a dense FFN layer and one
per-expert MoE expert are the same assembly, and a second implementation
was written and then deleted before it could drift from the first. That
function gains shape validation and a corrected doc comment — it claimed
gate/up were "interleaved" when they are concatenated, which matters
because reversing the order swaps every GLU's halves with no crash and no
size change. The order is now pinned against the same split the consumer
performs.
Partial layers fail closed: expert 0 present but expert 5 absent is an
error naming both counts, not a short entry list that silently drops
experts routing will later select.
The wider lesson, recorded in the E0 write-up: verify, slice and walk all
pass on an index that cannot serve, so decode stays a first-class row in
the preservation matrix rather than being inferred from verify.
E0 asserts that adding VINDEX3 support changes nothing observable on any VINDEX2 path. Without a fixed record, that degrades into "the two binaries agree with each other" — a condition any bug present in both satisfies, and close to circular given both are builds of the same binary. The baseline has to predate the v2 code and be committed. scripts/e0-capture-goldens.sh captures, from a pre-v2 binary: verify verdicts, index.json version fields, WALK top-K over three fixed prompts, greedy decode over the same three, and all eight slice presets. Paths, durations, throughputs and timestamps are normalised out, so a diff means behaviour rather than environment. A non-zero exit is recorded as part of the golden — a path that starts *succeeding* must be flagged too. Two corpora, both captured from 302ae59: gemma4-26b-a4b 18/18 clean. Decodes ("Paris.", a Fibonacci explanation, "100°C (212°F) at standard sea level"). olmoe-1b-7b 17/18 clean; decode panics, pinning the separate-tensor expert-store bug as a golden so its fix shows as a diff. The corpora themselves are NOT committed — they are regenerable, so PROVENANCE.txt pins the recipe instead: baseline commit, model repo, revision hash and exact extract flags. An earlier draft claimed the baseline expired at merge; it does not, since the checkpoint still extracts and the commit stays checkoutable. The reason to build the corpus early is that E0 is specified as continuous — it should be catching regressions while later V2-0 work touches shared code.
Three amendments, all from contact with the code and all recorded in place rather than silently applied. E0's corpus was specified as "existing production v1 indexes". There are none on the rig, so the experiment had no subject and every check would have passed by having nothing to check. Replaced with a constructed, pinned corpus: C1 a fresh Gemma extract, C2 pulled hub artifacts, C3 golden outputs from the pre-v2 binary. C1 must be `--level all` — the `all` slice preset needs lm_head and `router` needs router weights, so a lower level would silently reduce preset coverage to whatever the extract reached. Also corrected: eight slice presets, not six. E7's W0 control does not cover what E7 is about. Measured on the C1 extract, VINDEX2's searchable index holds 2,112 features per layer — the dense FFN width — against the 90,112 the expert population would contribute. The expert weights decode fine; they are not in the searchable surface. W1/W2/W3/W4 are about expert regions and W0 has none, so comparing them measures a coverage difference wearing a parity result's clothes, and "identical top-K rankings" is unevaluable. Resolved by restating the claim rather than rebuilding the control: expert-feature search is a NEW CAPABILITY of VINDEX3, not parity with VINDEX2. E7 now splits into a parity arm (W0 vs W1, dense model, genuine like-for-like) and a capability arm (MoE, correctness against directly- computed gate dot products, no baseline because none exists). Prior 9 splits to match. The rejected alternative — building an expert-bearing VINDEX2 index to serve as a control — would have been a control made for the experiment rather than the thing that shipped. The generation rename is reflected throughout, with a §12 note on why the LYRW sequence trails the container one and must not be "aligned".
…ects The physical index stores tensor regions; this manifest gives them meaning. Keeping the two apart is what makes "the binary says programme 4, the manifest says gpt-oss-expert-v1" unrepresentable rather than discouraged — LYRW carries no programme identity, so the manifest is the sole binding. Programme registry (§8.4) with frozen ids, each declaring the region roles it needs. Requirements are modelled as ALTERNATIVES rather than a flat set, because `gate + up` and `gate_up_fused` are equally valid ways to satisfy the same programme — which is exactly the V2-1 acceptance requirement that fused and decomposed storage "produce identical results under one manifest". `missing_roles` reports the closest alternative, so an index holding gate_up_fused is told it needs `down`, not that it needs gate, up and down. Router vocabulary sized against the four envelope routers, which disagree on nearly everything: softmax vs sigmoid scoring, plain vs grouped top-k, renormalise, norm-after-top-k, gate bias, route scale, quantile balancing, and Inkling's shared-expert sink. All four round-trip through one type. That is the falsifiable claim: a real model needing a knob that is not here is an ABI finding, not a post-freeze patch. The sink is not cosmetic — with it, shared experts draw from the same normalised weight budget as routed ones, so a reduction that ignores it silently over-weights the routed branch. A sink declared without a shared bank is therefore a defect, not a no-op. Per-layer, deliberately. A global `first_k_dense_replace` covers Kimi-Linear's leading dense layer and then fails on Inkling-Small, whose dense MLP sits mid-stack at index 2. Dense layers are expressed by absence from the manifest, which handles arbitrary schedules for free. Manifest-level checks are the ones answerable without reading a weight byte: unknown programme, latent bank without transforms, input_space contradicting its programme, selection exceeding the bank, sink without a shared bank, duplicate layer, unsupported schema version. Operand-absence needs the LYRW files and lands with the capability check. One root cause yields one defect: an unknown programme does not also report a space contradiction, or the reader chases input_space while the real problem is the name.
Five corrections from review of the manifest layer. Defect suppression was scoped correctly in the code but its TEST enforced over-suppression: an unknown programme was asserted to yield exactly one defect in total. That would have forbidden reporting independent defects alongside it, so a typo'd programme name could conceal a real second corruption. Suppression is now documented and tested as dependency-scoped — an unresolvable programme suppresses only checks whose interpretation depends on knowing the programme (input-space compatibility), and the assertion is by absence of that specific defect rather than by a count. Storage-shape checks added precisely because they must survive that suppression: empty storage reference, zero experts, zero-width dims, and two banks naming one storage location. None is a function of programme identity. Two banks both blank report emptiness rather than duplication — "duplicate storage ''" would point at the wrong fix. The sink asymmetry is now pinned in both directions. `shared_expert_sink` requires a shared bank; a shared bank does NOT require a sink. Kimi-style always-active shared experts outside sink normalisation are valid, and the flag must never become shorthand for "this layer has shared experts". Programme diagnostics gain determinism and self-description. Ties among equidistant alternatives resolve by declaration order — the alternatives are a fixed &'static slice and `min_by_key` keeps the first minimum, so the chosen layout cannot drift as iteration changes. `Unsatisfied::describe` names the layout it judged against: "closest accepted layout: gate_up_fused + down; missing down". A diagnostic that moves is a diagnostic nobody trusts. `eager` is retired in favour of `direct`, matching §6.2's normative encoding (none | direct | strided). It was a collision between §6.2 and §15.2 and is now gone from the enum, its serialised name, its flag mapping and every test. Fixture D is frozen: hidden 512, latent 256, expert intermediate 256, 112 experts, top-16, 2 shared. 448/224/192 broke Q4_K superblock alignment on every matrix (pad 224→256 is +14%, 192→256 is +33%), making the byte ledger a padding artefact; no proportional downscale fixes it. 112 = 7 x 16 divides the group width where 56 would bake a partial group into the fixture, and group width dividing segment width is a §7 rule, not a preference. Exact-K3-scale fidelity stays where it belongs: the byte-faithful bank at 896.
…ates
Preparation for the single traversal that authority derivation, operation
admission and kernel binding will all read from. Three independent
inspectors is how permissive logic gets in: each is individually
reasonable, and the union of their leniencies is what ships.
`satisfied_alternatives` replaces `satisfied_by`. A bank can physically
satisfy BOTH `gate + up + down` and `gate_up_fused + down`, and the kernel
registry may support one at a higher maturity than the other. Returning
only the first match would let a Production fused kernel hide behind a
Reference decomposed path — correctness-preserving, silently slower, and
miserable to debug. Traversal now preserves every usable layout and leaves
ranking to kernel binding. Declaration-order tie-breaking stays where it
belongs: the *unsatisfied* diagnostic, which needs one stable answer.
OperandCapability keeps four states that must never collapse:
Absent(kind) hard refusal; never a reference-executor fallback
PresentUnsupported bytes intact, codec/packing unreadable by THIS build
ReferenceExecutable correct execution, optimisation maturity incomplete
KernelExecutable specialised path, with its rung and kernel id
The middle two are the pair §11 legislates about. If "no grouped kernel"
and "operand absent" look alike, the reference executor becomes a fallback
for corruption rather than for immaturity, and an index that should have
been refused gets served slowly instead. PresentUnsupported also carries
forward §6.5's parse-time rule: bytes this binary cannot interpret are not
missing bytes — the artifact may be intact and simply newer.
AbsenceKind distinguishes four situations with similar execution outcomes
and different fixes: absent from every segment (extract it), partial
segment coverage (complete the transfer — it exists), incompatible segment
sets between two roles (reconcile the selection, not the storage), and
omission by the active profile (not a defect at all — a browse slice has no
`down` by design, and reporting that as corruption teaches operators to
ignore diagnostics).
RegionCoordinate carries {layer, bank, role, segment} per §11. Segment
identity stays individual even for adjacent failures: presentation may
compact a run, the report must not, because a consumer re-fetching exactly
the broken segments needs the list. `Some(0)` and `None` are deliberately
distinct — segment 0 is one of several, None is an unsegmented bank.
RegionRole gains Ord by registry tag so reports sort in spec order and stay
diffable.
… type Authority derivation (§9.2) with the invariant enforced rather than documented: replacing a selected component with weaker fidelity, or removing an operand, can never increase any derived authority. The fold is three stages and deliberately boring — weakest selected region fidelity, capped by operation completeness, capped by declared structural omission. Its input struct contains no names, paths or profile identity, so the fold *cannot* consult a filename or a profile name even by accident; that is a type-level guarantee rather than a code-review one. Kernel maturity is not an input. A Production kernel and the reference path executing the same bytes have identical fidelity — conflating them would let a slow-but-exact path present as approximate, or worse, a fast approximation present as exact. Monotonicity is tested exhaustively rather than sampled: the lattice is five levels, so enumerating every (before, after) pair over every input axis is cheap and total. Four properties hold across the whole space — weakening a region, adding a region, losing execution completeness, and declaring a structural change all move authority down or leave it alone. Order-independence is pinned separately, because a future "first wins" shortcut would silently break weakest-link while passing every other test. Two floors worth stating: an empty selection derives analysis-only, not source-exact — nothing selected cannot be exact, and defaulting high would make the emptiest profile the most authoritative. And capping is always a minimum, never a floor, so a structural change cannot *raise* an already-weaker selection. `DerivedAuthority` reports which cap bound it, so "why is this only analysis-only" is answerable without re-deriving. BankSelection is traversal's input — what variant-and-segment resolution produces, not something traversal computes. Fidelity is carried per region and never inferred from the format tag: a Q6_K container of native MXFP4 values is source-equivalent while Q6_K quantised from BF16 is numerically-approximate, and the tag alone cannot tell those apart. It also records roles the profile deliberately omits, separately from roles that are absent, because a browse slice without `down` is working as designed.
…l result
Segment incompatibility is relational — it is true of no role individually
— so it does not belong in the operand model. `IncompatibleSegmentSet` is
removed from `AbsenceKind` and replaced by `SegmentCompatibility` on the
alternative evaluation.
The architectural split is now explicit:
OperandCapability what each selected region can provide
SegmentCompatibility whether those operands can participate in ONE
computation
(execution derives from both)
Assigning the relational fact to each operand separately would state the
symptom twice and the cause nowhere.
Three verdicts, with the precedence they imply:
Compatible every required role covers the required set
Partial every role covers the SAME set, short of required — no
cross-role disagreement, the selection is consistently
incomplete
Incompatible roles cover different sets, shaped as NoCommonCoverage
(nowhere the computation could run) or UnequalCoverage
(part of the population is executable, the selection is not)
Both incompatible shapes admit identically but are different situations,
and a reader deserves to know which.
Subsumption, not suppression: `subsumed_per_role_gaps` retains the
role-local partial-coverage facts as evidence on the finding, so machines
and verbose inspection keep them while the user-facing diagnosis names one
cause. The information does not disappear; the report selects the more
explanatory parent.
Compatibility is computed as a GLOBAL intersection, not pairwise. Three
roles covering {0,1}, {1,2} and {2,0} agree pairwise and share nothing
globally; a pairwise check would have called that compatible.
`is_invalid_selection` marks incompatibility as fail-closed. It is not a
declared approximation policy and must never be laundered into
structurally-approximate authority: a deliberately omitted `down` in a
browse slice is intentional and derives analysis-only, whereas two roles
accidentally resolving to disjoint populations is a resolution error. The
authority fold must not be asked to legitimise a contradictory selection.
Traversal answers one question: are the selected bytes sufficient for this
programme? It stops at reference executability. It does not choose a kernel,
does not decide authority, and does not know what a profile is called.
Four-tier precedence, with U the required population and Cr each required
role's usable coverage:
1 any Cr = empty role-local failure; compatibility MOOT, not evaluated
2 all Cr non-empty, incompatible segment sets
sets differ
3 all Cr equal, != U consistently partial coverage
4 all Cr = U reference-executable
Tier 1 outranking tier 2 is load-bearing: asking whether two roles' segments
agree is meaningless when one of them has no segments. That is expressed in
the type — `compatibility` is Option, None meaning "moot", not "compatible".
"Usable" excludes regions this build cannot interpret, so an all-unsupported
role lands in tier 1 naming the codec rather than being mistaken for a
coverage gap.
Two conflations were removed rather than worked around:
OperandCapability combined physical state with kernel maturity. Traversal
stops at reference executability, so that type carried a variant traversal
could never emit. Split into role::{OperandAvailability, ReferenceSupport}
with maturity moved entirely to kernel.rs — a traversal that could name a
kernel would be a traversal that had already chosen one.
AbsenceKind::PartialSegmentCoverage stated the same fact as
SegmentCompatibility::Partial at the wrong level. Partial coverage means
Cr != empty, which is tier 2/3 by definition, so the role-local variant was
a second representation inviting the two to disagree. Removed; usable
coverage plus the compatibility verdict carry it.
Alternatives are evaluated independently and every successful one is
preserved. A bank incompatible under gate+up+down and complete under
gate_up_fused+down is executable, and the failed alternative is evidence
rather than a layer defect. A closest failure is chosen only when none
succeed, ranked by fewest unusable roles with declaration order as the
tie-break.
The decisive test is provenance: declared and undeclared absence produce the
same physical shape and the same admission result, and must differ in
diagnosis. A gate-only slice that declared its omissions reports no defect;
a gate-only index that lost its regions reports one. Nothing infers intent
from missing bytes.
Three questions kept apart, per review:
can this operation run? admission (this commit)
how faithfully would it run? authority, folded over the regions THAT
operation consumes
which implementation runs it? kernel binding (later)
`OperationCapability` is admission plus an OPTIONAL authority, and the
optionality carries the load. An unavailable operation has no fidelity
because it does not run. A contradictory selection also has none — and must
not be handed analysis-only or structurally-approximate merely because those
are low lattice values. Assigning a weak-but-valid authority to an invalid
selection would launder a refusal into a downgrade, and a caller asking "is
this at least analysis-only?" would wrongly proceed.
`OperationPlan` carries the regions an operation consumes, which is what
makes per-operation authority possible: the fold's domain becomes the
operation's own regions rather than the whole index. An index with exact
gate rows and approximate down regions can then honestly report WALK as
source-exact and local decode as numerically-approximate.
Failures distinguish causes an operator triages differently — unusable
region (with exact coordinate), invalid selection, missing document input,
missing contract, no executable route — and a test requires no two of them
to render alike. `MissingContract` exists specifically so remote execution
can never be inferred from local absence: "routed operands missing" and
"routed branch deliberately remote" are the same bytes with different
meanings, and only a declaration can tell them apart.
Degradation is separate from failure. Missing query metadata reduces label
richness and never affects correctness (§15.3), and the wording says so —
an operator must not read it as "results may be wrong".
Also restores a tier-1 cause the traversal was collapsing. Regions present
only outside the required population were being reported as
AbsentEverywhere, which is the wrong repair: the bytes are real and the fix
is to reconcile the selection's segment set, not to extract a variant. Now
`PresentOutsidePopulation`, naming both what was found and what was
required.
…e split Pins the COMPILE contract: COMPILE performs document-scoped canonical reconstruction from declared baseline variants. It is independent of the active profile and reports fidelity derived from those baseline regions. Scope is structural, not documented. `CapabilityReport` splits into `DocumentCapabilities` and `ProfileCapabilities` so callers cannot assume every operation describes the resolved profile — because one of them does not. This combination is now legible rather than surprising: profile.local_decode unavailable document.reconstruct_canonical available A browse profile selecting only gate regions fails local decode while its parent document, holding every baseline, reconstructs perfectly. The mirror case is equally coherent and equally tested: baselines lost, profile variants intact — serves fine, cannot be exported. `CanonicalSelection::from_catalogue` takes the representation catalogue and nothing else. No profile, no placement, no browse mode. Reusing the profile's BankSelection and overriding fields would mix two scopes and let a deliberate serving omission leak into an export operation that has nothing to do with serving. Canonical is not a fidelity claim. The baseline is canonical *within the document*; its fidelity is still measured against the source checkpoint. So `reconstruct_canonical: available, authority: numerically-approximate` is a valid result, and the operation is deliberately not named ReconstructCheckpoint or ReconstructOriginal — those promise a fidelity the baseline may not possess. The sharpest test: a region set whose baseline is source-EQUIVALENT beside a sibling that is source-EXACT still reconstructs from the baseline. Canonical means declared, not best available. And an absent baseline never falls back to a present sibling, because falling back would silently change what COMPILE emits. `TensorReconstruction` targets the checkpoint's tensor vocabulary rather than the index's storage vocabulary: a fused region may split into two checkpoint tensors, separate roles may join into one, and values/scales recombine. The mapping comes from importer metadata, never from guessing at role names — otherwise COMPILE reproduces storage structure and calls it a model. Bounded to four recipes; not a graph interpreter. Exporting exactly what a profile selected remains a *different* operation and should not be called COMPILE or reconstruct when it lands.
An operation can run more than one way, and the ways need not be equally faithful. WALK may read a direct gate region or stride the gate half of a fused one — different bytes, possibly different fidelity. Attaching one authority to the abstract operation forces a bad choice: report the stronger and binding may later pick the weaker, making the report a lie; report the weaker and an exact route is understated; pick during inference and binding is constrained before it has seen the kernel registry. So `OperationCapability` now carries `routes: Vec<QualifiedOperationRoute>` and has no authority field at all. Authority is reachable only through a route, and the accessors are named `best_achievable_authority` / `worst_achievable_authority` — achievable, not achieved, because no route has been bound. `authority_is_settled` says whether binding can still move the answer. The fail-closed invariant survives in a stronger form: routes are empty exactly when the operation is unavailable, so no routes means no authority and a contradictory selection cannot acquire a weak-but-valid fidelity by default. `is_well_formed` states that equivalence and a test enforces it. Plans are fixed requirements plus independent choice groups, not a Cartesian product. Thirty layers each admitting two alternatives is 2^30 whole-model routes; enumerating them is not a representation. Binding picks one alternative per group, and before that the honest statement about such a plan is a range rather than a value. A test builds thirty choice groups and asserts the representation stays thirty. Alternative identity is carried into the plan so kernel binding can match a kernel to the layout it actually supports, and `best_alternative` ranks by weakest-link rather than by "has an exact region somewhere" — a uniformly approximate route beats one that is exact in two places and structurally approximate in a third.
… residual Two things pinned before inference, because both would let `infer_walk` answer an underspecified question correctly only by accident. WALK capability needs a target. "Is WALK available?" has no answer when one bank has a direct gate route and another is serving-only with browse mode `none` — both yes and no are defensible until someone says which bank they meant. `WalkRequest` therefore carries a `WalkTarget`, and `AllWalkableBanks` means the declared searchable surface, deliberately not "whichever banks happen to work". Unwalkable requested banks fail the request by default. Absent query metadata reduces label richness and leaves rankings correct, so it degrades; a missing searchable gate population changes the RESULT SET. Silently dropping a bank and returning a global ranking over the remainder would be a wrong answer wearing a successful one's clothes. `PartialPolicy` defaults to RequireAll and partial results are opt-in, with the dropped banks reported so a caller knows the ranking's true scope. `WalkInput::ResidualVector` means the model's residual space, never a bank's latent space, and width is checked against the residual dimension rather than against the bank being searched. The trap this closes is specific: K3's latent banks are 3584 wide, so a 3584-wide query would dot-product against latent gate rows perfectly happily while skipping `routed_input` entirely — succeeding at a different question. The diagnosis says so in as many words. `TextQuery` is a separate input mode so a caller supplying a prebuilt vector is not refused for lacking tokenizer and embedding infrastructure it never uses. Also records, on `PlanChoice::best_alternative`, that it is for introspection and planning only and must never serve as a binding tie-break: when two alternatives carry equal authority, kernel binding has to stay free to choose on support and performance grounds, and an authority-oriented tie-break would silently decide a question belonging to the kernel registry using a criterion that cannot distinguish the candidates anyway.
WALK inference, built to satisfy one negative acceptance test: a WALK report must be derivable without ever asking whether the model can perform an expert forward pass. The strongest form that property could take turned out to be structural rather than behavioural. `WalkableBank` carries a browse mode, gate accesses and an optional latent transform — and nothing else. There is no `down` field to consult, no router, no expert traversal, no kernel maturity, no query metadata. Unreadable expert regions cannot perturb a WALK report because they are not reachable from the function's inputs. Scope is enforced, not assumed. With bank 0 walkable and bank 1 serving-only: target bank 0 available target 0 and 1 unavailable, naming layer 1 and its browse mode target all unavailable, naming layer 1 That third line is what stops "some walkable content exists" becoming "WALK works". The declared surface is a list, never a filter over health — a bank declared searchable but absent from the index is reported as a failure rather than quietly dropped, which is the back door that would have recreated working-subset semantics through implementation. Partial results are opt-in and do not lower authority. Bank A's gate rows are still source-exact when bank B is dropped; the population shrank, the numbers did not degrade. `ResultSetCompleteness` therefore sits beside authority rather than inside the fold, matching the distinction already drawn for absent query metadata. Latent banks put `routed_input` in the plan, so exact gate rows are not enough on their own — weakening the transform weakens latent WALK and leaves residual WALK untouched. A latent-width query vector is refused even though it would dot-product against latent gate rows perfectly happily, because succeeding there means having skipped the projection. One choice group per target bank, never a Cartesian product across banks: eight banks produce eight choice groups on one route. `ResidualVector` needs no tokenizer and no embeddings; `TextQuery` needs both, and folds embedding fidelity into route authority while the tokenizer contributes availability only — it has no numerical fidelity to contribute.
Preparation for local decode, and a correction to what I proposed. I was going to put embeddings, norms and attention weights into an environment analogous to `TextQueryInputs`. That would have blurred three things which must stay distinguishable: what the index contains the catalogue what the profile selected the resolved selection what the caller supplied the request / contract Stored weights belong to the second. An environment holding both stored tensors and caller inputs would leave the loader unable to say whether a missing tensor was never extracted, deliberately dropped, or simply not passed in — which is precisely the provenance distinction the rest of this layer works to preserve. So `SelectedComponent` spans both addressing schemes: bank regions and manifest-addressed tensors. `ComponentCoordinate` likewise, carrying the weight class, optional layer and tensor key for the manifest side. Only bank regions have a role, which the type states by returning `Option`. `SelectedTensor` keeps omission provenance separate from readability. A deliberately dropped LM head in an attention-only client slice is unusable and not a defect; the same tensor missing without declaration is unusable and a defect; an unreadable codec is unusable, not a defect, and possibly just newer than this build. Three states, three repairs. `OperationPlan::fixed_regions` becomes `fixed_components`, with a `fixed_regions()` constructor retained for the WALK shape — WALK genuinely reads only bank regions and a latent transform, and should not have to mention a component type it never varies. No behavioural change: the authority fold now sums fidelity over components rather than regions, and both kinds contribute identically.
Assembly, as intended. The adapter says which tensors and banks a layer
needs; programme traversal says which bank-region arrangements execute; this
joins them and folds authority. Kernel maturity is never consulted.
The two decisive tests are inverses, and together they prove neither side
pretends to validate the other:
experts complete, routed_input absent
→ bank traversal succeeds
→ local decode fails, naming routed_input
fixed path complete, down unreadable
→ every tensor resolves
→ local decode fails through bank traversal, naming the bank
A complete expert bank is not a decodable layer. Traversal answers whether
the BANK COMPUTATION runs; requirements answer whether the WHOLE LAYER PATH
does. Router weights, latent transforms and routed output norms are fixed
dependencies of the layer, not part of the bank's programme.
Four things pinned in the requirement model:
Requirements never restate the programme. An adapter names participating
banks and surrounding tensors; gate/up/down alternatives stay in the
registry, so a programme change does not require synchronised edits to every
adapter.
Requirements carry alternatives, because the fused/decomposed problem recurs
outside expert regions. A tied LM head is the case: some models store a
dedicated tensor, others reuse the embedding table, and a flat mandatory
list would reject a valid model. Same rule as programme alternatives —
declaration order decides which failure is reported, never which success is
used.
"Present and readable" is not enough. `ComponentUsability` adds
ContractMismatch beside absence and unsupported encoding, because a readable
tensor of the wrong shape is neither a missing file nor a build limitation —
it is an invalid index or an adapter mismatch, and a wrong-but-compatible
shape can survive a long way down a generic buffer path or never fail at
all. Five states, five repairs.
The request is one variant. Residual-to-logits and layer-range boundaries
change which fixed components are required, so each deserves its own variant
rather than a general request whose optional-field combinations become
ambiguous. TokenIdsToLogits needs no tokenizer: the ids are already
supplied.
Failures aggregate. A missing router at layer 12 and an unreadable norm at
layer 19 are independent facts, and reporting only the first would make
fixing an index an iterative guessing game.
The dense schedule is a list of per-layer requirements, so Kimi-Linear's
leading dense layer and Inkling-Small's mid-stack one work through one
mechanism with no global field to get wrong.
…aught one E0 is specified as step zero, green throughout, any regression blocking merge. It had not been running. Twenty commits touched shared loading code with VINDEX2 compatibility in an unknown state rather than a preserved one. It now runs as a named CI step, separate from the general test run so a failure reads as "the shipped generation regressed" rather than as one red test among two thousand. ## Scope, stated rather than implied The full matrix — decode, WALK, slicing, sharding, publish/pull — needs multi-GB checkpoints and cannot run in CI. Those rows run locally against the committed goldens via scripts/e0-capture-goldens.sh. What runs in CI is the generation-boundary subset, which needs no weights and is precisely the part VINDEX3 work puts at risk. ## The regression it caught, first run `v1_config_loads_with_defaults` broke: `index.json` version 1 no longer loaded. My generation model assumed a bijection between container generation and index.json.version. There isn't one below 2. Version 1 is a *legacy schema of the same shipped generation* — such indexes exist in the wild, and the loader has always read them by filling absent fields with defaults. Treating the version as a generation identifier rather than a generation floor refused them outright, breaking the exact compatibility that dual-generation support exists to protect. Fixed: versions 1 and 2 both resolve to VINDEX2, 3 to VINDEX3, and the supported-versions diagnostic now reads "1-2 (VINDEX2), 3 (VINDEX3)". The generation is named for its *current* index.json version, and the docs say so along with why the mapping is not a bijection. That is what a preservation matrix is for. Caught on the introducing branch it is a one-line mapping fix; caught after another ten changes it would have been archaeology. 2031 tests, 0 clippy warnings.
The E0 regression exposed a model error, not a forgotten branch:
index.json schema version != container generation
VINDEX2 spans schemas 1-2; VINDEX3 begins at 3. The mapping is many-to-one.
So the types no longer permit conflating them: `IndexSchemaVersion` is a
newtype, and no API accepts a bare u32 and calls it a generation. The
previous code compiled perfectly while making exactly that mistake.
Four statements are now independently true and independently testable:
VINDEX2 writes schema 2
VINDEX2 reads schemas 1-2
VINDEX3 writes schema 3
VINDEX3 reads schema 3
`generation_for_schema` maps a revision to its owning generation;
`current_schema_version` and `supported_schema_versions` answer the two
questions separately. A test asserts no two generations claim the same
schema, because overlap would make dispatch ambiguous and falsify the
sole-discriminator property.
Spec §12.1 amended to match. `index.json.version` remains the sole dispatch
input — no filename sniffing — but it is a *schema* discriminator, and the
loader maps revisions to generations. The table is written out, along with
why a generation is named for the schema it writes rather than the only one
it reads.
Unified dispatch and direct-loader refusal are separated and both tested:
schema 1 routes to the VINDEX2 loader, and the VINDEX3 loader still refuses
it by name.
## E0 now has two named gates
"E0 green" must never be reported when only part of it ran:
E0-CI generation/schema boundary, weight-free — required merge check
E0-FULL checkpoint-backed matrix against committed goldens — local,
reported as "green at commit <sha>"
The reporting convention is written into the experiments document so a
future status line cannot claim the whole matrix on the strength of the
subset. A synthetic VINDEX2 artifact exercising load/verify/slice and one
trivial decode in CI is noted as the way to narrow the gap.
2032 tests, 0 clippy warnings.
`resolve_requirement` returned on the first satisfied candidate. A document holding both a dedicated lm_head and a usable embedding table has two valid routes, and returning the first settled the route — and therefore the authority and the kernel choice — before binding had seen the registry. The same collapse already fixed for expert alternatives, recreated one layer up. My own commit message claimed the opposite property held; it did not. The rule now: 0 usable candidates → unavailable, naming EVERY candidate's own repair 1 usable candidate → fixed binding, no degenerate choice group 2+ usable candidates → independent choice group Failed candidates survive even when another succeeds. An operator debugging why the tied head was used needs to see that the dedicated one was absent, and "lm_head unsatisfied" alone says nothing about which to fix. Choice groups therefore stop being bank-only: `QualifiedAlternative` gains a components side so a requirement's candidates can form a group beside the expert banks, and the authority fold reads both. Also adds the identity split this needs. Variant must NOT be part of physical identity — folding it in would defeat the deduplication it exists for, since two catalogue declarations may name one extent: RepresentationIdentity which declaration was selected PhysicalStorageId which bytes that resolves to same bytes, two semantic uses → one residency allocation same bytes, two permitted views → one component, two bindings same logical tensor, duplicated bytes → two components Only exact extents deduplicate; partial overlap stays distinct, because a sub-range is either legitimate or a catalogue error and neither makes it the same component. Two declarations sharing an extent while disagreeing about it are a catalogue defect, not two components — the bytes cannot be both. `ComponentView` records how bytes perform a role, because embedding-as-lm-head is a transpose, not the same access. Contracts are checked AFTER the view: a dedicated head is [hidden, vocab] and an embedding is [vocab, hidden], so comparing the requirement against raw storage would reject a valid tied-weight model. No view alters values, so views never touch authority — they affect kernel eligibility and access efficiency, and a route needing transposed access must be exposed at Reference maturity rather than silently repacked at decode time. 2055 tests, 0 clippy warnings.
Fixture A executes through a bound VINDEX3 operation and reproduces an
independently implemented oracle to below 1e-6, from both decomposed and
fused storage, agreeing at every internal checkpoint.
Scope, precisely: this is V2-1's first successful rung, not the gate. It
does not touch Vindex::open, on-disk profile resolution, Gemma, shared
experts or latent K3 execution.
## The boundary
BoundMoeOperation { router, transforms, banks, reduction } is the terminal
product of binding. No manifest lookup, capability traversal, authority
fold, variant resolution or kernel search is reachable from execute() —
resolution that stays reachable gets called, and per-token cost starts
tracking catalogue size rather than work size.
The router is a BoundRouter rather than a bare tensor: top-k depth,
renormalisation and per-expert scaling decide which experts run and by how
much, so leaving them for the token loop to infer would breach the same
boundary. Validation lives on the operation and is called by binding.
BoundProjection resolves fused vs decomposed once, behind an interface that
yields the same two rows either way, so activation, the gated product, the
down projection and the reduction cannot acquire a layout assumption.
## Instrumented, not final-vector-only
Checkpoints: router scores, selection margin, selected ids and order, gate
weights, per-expert outputs, reduction, residual delta. One generic
execution over a TraceSink — NoTrace compiles away, so the fixture exercises
the code Gemma will run rather than an instrumented copy.
Mutation-checked: interleaved fused reads, activation on the product rather
than the gate, and a transposed router each fail the tests that should catch
them and no others.
## Tie behaviour, pinned before Gemma
The incumbent's sort_unstable_by over partial_cmp is unspecified for ties.
This path commits to score descending, then expert id ascending, using
f32::total_cmp — a real total order, not a fallback to Equal. Non-finite
scores are refused rather than ordered, since a NaN would otherwise be
selected or dropped by sort mechanics.
top_k_with_margin reports score[k-1] - score[k] so a Gemma divergence at an
exact tie is triaged as a selection-policy difference in minutes rather than
mistaken for a weight-decoding fault. VINDEX2's production behaviour is
unchanged.
## Measurement
benches/vindex3_bound_execute — successor to vindex_storage_dispatch, gating
"has resolution leaked into decode". Its first draft asserted flat cost in
population; measuring said otherwise, and the claim was wrong — routing must
score every expert. Restated as a growth-shape budget: 64x population buys
~1.9x, which is the router term.
It caught a real defect: decode_at rebuilt its operand name per element,
allocating a String on every scalar read.
before after
population 64 2.42 ms 142 us
decomposed/fused 43% apart 0.7% apart
trace overhead - 0.4%
The 43% "arrangement" gap was the allocation, not the layout —
gate_up_fused is a longer name than gate. Not a VINDEX3 speed result; the
removal of an accidental allocation catastrophe from new code.
tests/vindex3_allocation_guard — thread-local counting allocator, scoped to
that test binary. Invariant: operand lookup and scalar access must not
allocate in proportion to dimensions, population or elements read. Tightens
to zero per token when execute_into lands.
examples/vindex3_residency_probe — successor to mmap_cold_read_probe, via
mincore rather than fault counting. The bound plan predicted its resident
set exactly: 200 pages predicted, 200 resident, zero overshoot, 1.63% of a
192 MiB layer resident after one token.
Found that msync(MS_INVALIDATE) before madvise(MADV_DONTNEED) does evict on
Darwin, where MADV_DONTNEED alone leaves a freshly-written file 100%
resident. mmap_cold_read_probe works around the same limitation with
F_NOCACHE pread.
## fix: larql verify was nondeterministic
A real product defect, found by E0-FULL and unrelated to VINDEX3.
`verify_checksums` iterated a `HashMap` directly, and Rust randomises
`HashMap` iteration order per process, so `larql verify` printed its findings
in a different order on every run against an identical, intact artifact —
three invocations, three digests. That makes it unusable for goldens, CI
diffs, signed reports and operator triage, and it made E0's `verify` row
permanently unfalsifiable.
Findings are now rendered in canonical artifact order. Sorted at the
collection boundary rather than in the CLI, so every caller inherits it and
hashing stays free to run in any order — including in parallel later —
without scheduling deciding user-visible output. Five consecutive
invocations now produce one digest.
Five tests, mutation-checked. Two of them fail without the sort; the other
three do not, and one of those was actively misleading:
`repeated_verification_of_one_artifact_is_byte_identical` passed even with
the fix removed, because per-process hash randomisation means one map
iterates identically within a single test. It could never have caught the
bug it was named for. Renamed to what it actually guards (no state carried
between calls), with the limitation documented, and
`insertion_order_does_not_affect_output_order` marked as the load-bearing
guard.
## E0-FULL now has a runner
The goldens were committed and provenance-stamped, and nothing ever read them
back — an assertion no one was making. `e0-capture-goldens.sh` was
capture-only.
Added `e0-verify-goldens.sh`, which replays the record against a current
binary and diffs every row; exit status is the verdict (0 match, 1 differ, 2
goldens absent). The corpus — prompts, row set, normalisation — moved to
`scripts/lib/e0-corpus.sh` and is shared by both scripts, because a
capture-side and verify-side copy would drift and E0-FULL would degrade into
"the two scripts agree with each other", the same circularity the capture
script's own preamble warns about.
Mutation-checked against a stub binary: a changed decode path fails exactly
the three decode rows and exits 1; unchanged exits 0; missing goldens exit 2.
Two harness bugs fixed along the way, both of which produced confident false
positives that read exactly like the regression under test:
- Replay parameters were defaulted rather than read from the golden's
PROVENANCE.txt. Replaying 24 tokens against a 16-token golden differs on
every decode row.
- The timing normaliser ended in `\b`, a GNU extension that BSD `sed -E`
neither supports nor complains about — it silently never matched, so no
duration was ever stripped on macOS. Replaced with a portable form;
verified that durations strip while file sizes, which are behaviour,
survive.
Rows whose output is a set rather than a sequence (`verify`) are canonicalised
on both sides, so a golden captured from the still-nondeterministic baseline
binary remains usable. WALK rows are deliberately excluded — their content is
a ranking.
## E0-FULL status: not discharged
Run against a freshly extracted C1 (gemma-4-26B-A4B-it @ 7d4c97e5, the
revision the golden recipe pins):
matched 5 verify, decode_p0, decode_p1, decode_p2, index_version
differed 12 show, walk_p0..2, slice_* (8)
All three greedy decode rows are token-identical, and all 632 WALK ranking
lines match. That is strong evidence VINDEX2 execution has not regressed.
The 12 diffs are traced, one line each, and neither cause is a regression:
- show + 8 slice rows: `index.json 5.9 KB` vs `5.7 KB`. I extracted with the
current binary instead of rebuilding the baseline at 6eae5ea as the recipe
prescribes, so this conflates extractor changes with reader changes — the
precise confound E0 exists to prevent. My procedural error.
- walk x3: the timing line only, because the goldens were captured with the
broken normaliser and hold literal durations.
Both need the prescribed baseline reconstruction (baseline binary → baseline
extraction → current reader against that artifact), which is a separate
preservation change. Status stands as:
E0-CI green
E0-FULL decode rows green
E0-FULL remaining rows not discharged
## Also
- larql-compute: f16_to_f32 made public. It is the only correct f16 decoder
in the workspace — its subnormal branch fixes a 2x error that a from-
scratch reimplementation reproduces almost every time.
- Fixtures are public, not cfg(test): tests, bench, demo and future
conformance checks share one definition.
- Fixture A's population moved 4 -> 5. With population and hidden both 4 it
could not have detected a transposed router.
2238 tests, 0 clippy warnings, no runtime file below 90% line coverage
(14 of 18 at 100%). Non-unix branch of the residency probe verified
standalone; cross-target cargo check is blocked by ring/openssl-sys needing
cross-compilers.
VINDEX3 has independently routed a real Gemma activation through real
checkpoint weights and selected exactly the same experts as production LARQL.
layer 5, hidden 2816, 128 experts, top-8
input: real h_post_attn, last token of "The capital of France is"
incumbent [17, 100, 126, 14, 120, 28, 73, 32]
vindex3 [17, 100, 126, 14, 120, 28, 73, 32]
gate weights max|delta| = 6.99e-4
boundary margin 0.000909
Same index, same Q4_K expert bytes, same f32 router, same activation. Only
the execution path differs, so a divergence would have been unambiguously
about execution rather than about re-extraction — which is why this binds
over the incumbent's own bytes instead of extracting a VINDEX3 container
first.
## Three corrections the real model forced
None was reachable synthetically. Each would have survived every shape check.
### Router input is not expert input
Gemma's `moe_router_input` applies a router-specific RMS norm, a learned
element-wise scale and a scalar on top of the vector the experts consume, so
routing scores a *different vector*. The operation fed one input to both. It
would have scored on the expert input and selected different experts — a
wrong answer with no shape error anywhere.
`MoeInputs::{shared, split}` expresses the distinction without making Gemma
special. Both vectors are supplied by the surrounding block rather than
derived here, for the same reason `execute` returns a delta: norms and scales
belong to the block, and deriving them would re-model the incumbent's routing
policy enums inside VINDEX3 with somewhere for the two to disagree.
Pinned by a test where the bank input selects [2, 0] and the router input
selects [1, 3] over identical experts.
### A shard is a subset of the routing universe
`validate` required `router.population() == bank.population()`. Wrong, and
`bank.rs` already documented the opposite: "expert 40 may be the first one a
shard carries." It would have refused every expert-server slice. The correct
invariant is that every stored expert id lies inside the router's address
space; equality is a separate question, now `holds_full_population()`.
Relaxing it created an execution requirement, so that is pinned too:
> If routing selects an expert absent from the bound local bank, execution
> must produce an explicit missing-expert result — not skip it, not
> renormalise around it, not substitute another.
`SelectedExpertNotResident` is a new variant, deliberately distinct from
`ExpertOutOfRange`: one means fetch the operand, the other means the index is
wrong, and collapsing them would leave an operator unable to tell which. It
names the expert, the layer and bank coordinate, and both populations.
Remote placement will later satisfy exactly this request without touching
router semantics.
Mutation-checked: making execution `continue` past a non-resident expert
fails five of the nine placement tests.
### Physical shape is not semantic operand shape
Q4_K pads the intermediate axis. Gemma stores `gate_up` at [2x704, hidden]
unpadded and `down` at [hidden, 768] — 704 rounded to a 256-multiple — so the
two regions disagree about the intermediate width.
`ComponentView::Slice` handled it with no index rewrite, no cropped copy, no
activation padding in the generic runtime, and no lying about the stored
extent: bind the stored [hidden, 768], let the role see [hidden, 704]. The
incumbent reaches the same place by zero-padding the activation instead.
Pinned with a poison-tail test: the padding columns hold 1e9, and the logical
view never observes them. Mutation-checked by swapping the view to Direct,
which fails three tests. A guard-the-guard test confirms the poison is really
in the stored bytes, so the assertions have something to catch.
## What is not claimed
Bit-exactness, and deliberately not a tolerance argument either. Two
structural numerical differences remain:
router incumbent BLAS sgemv vs index-order f32 accumulation
experts incumbent Q4_K x Q8_K integer dot vs dequantised f32 reference
A residual-delta tolerance would blend router accumulation order, softmax and
renormalisation, activation quantisation, integer dot rounding, expert
accumulation order and reduction order. Passing it would establish nothing;
failing it would identify nothing. So the comparison stops at checkpoint
three, where it stops being meaningful.
The 0.000909 margin establishes only that this particular selection was not
decided by tie policy. It is not a robustness margin across prompts or layers,
least of all while scoring accumulation differs.
## Next
Kernel binding, in two separate rungs: bind the incumbent router kernel and
require score/selection/weight identity, then bind the Q4_K x Q8_K expert
kernel and compare per-expert outputs, reduction and residual delta — aiming
for bit identity first, and introducing a tolerance only when the exact
floating-point operation that makes it impossible can be named.
2268 tests, 0 clippy warnings.
A VINDEX3 BoundRouter, consuming real VINDEX2 storage and a real Gemma
activation, calls the exact production scoring functions and reproduces every
observable routing stage bit-for-bit.
layer 5, hidden 2816, 128 experts, top-8
input: real h_post_attn, last token of "The capital of France is"
1 raw scores BIT-IDENTICAL
2 top-k id/score pairs BIT-IDENTICAL
3 boundary margin 0.00090900064
4 pre-norm weights BIT-IDENTICAL
5 normalised weights BIT-IDENTICAL
The margin is non-zero, so this selection was decided by the scores and the
result is a legitimate exact parity rather than a coincidence of tie handling.
## Binding, not reimplementing
`matmul_vec` and `softmax` are now public in larql-compute, and
`RouterKernel::Incumbent` calls those exact functions. Writing a BLAS-shaped
loop here and calling the agreement "parity" would have proved only that two
similar loops agree.
`BoundTensor::as_f32_slice()` hands over the stored bytes or refuses; there is
no path where a reconstruction quietly substitutes for the operand. A bridge
that dequantised into an incumbent-shaped temporary could have reached the
same numbers while proving nothing about the binding architecture.
`RouterKernel` is a bound choice like every other decision in the object, and
the reference kernel remains the default and the oracle.
## The ladder prevented a false diagnosis
Its first run read:
1 raw scores BIT-IDENTICAL
4 pre-norm weights BIT-IDENTICAL
5 normalised weights max|delta| = 6.994e-4
Scores identical, final weights not — so the fault was in post-processing and
could not have been the BLAS-vs-index-order accumulation that a single
end-to-end number would have been blamed on. The cause was a missing bound
operand: Gemma's policy is PerExpert and the harness had bound no scale, so
execution silently declined to scale.
## Tie ordering stays canonical, deliberately
The incumbent's top-k uses `sort_unstable_by` over `partial_cmp`, which is
unspecified for ties. That comparator is *not* bound: importing an unspecified
order into a path whose value is being a reproducible oracle would defeat the
purpose. VINDEX3 keeps score-descending then expert-id-ascending over a total
order, and ladder step 2 confirms the two agree whenever scores decide.
A future whole-model report should classify each layer as exact-parity
non-tied, tie-equivalent under differing tie contracts, or actual divergence,
so a legitimate canonical tie decision is never read as a kernel failure.
## Invalid scaling is now unrepresentable
enum BoundExpertScaling { None, PerExpert { scales } }
replaces a policy enum beside an `Option<BoundTensor>`, which permitted:
PerExpert + None policy demands a scale that was never bound
None + Some(scales) an operand nothing reads
The first is what this commit's own harness built, and it executed. Coupling
the policy to its operand is the difference between a check someone can forget
to call and a mistake that does not compile.
Scale coverage is defined over the **routing population**, not the resident
experts. Expert 90's learned scale is part of what routing means whether or
not expert 90 is resident, so a shard carries the whole vector; truncating it
would make two shards of one model weight the same expert differently.
Non-finite scales are refused at bind time, before a NaN can propagate through
the reduction into the residual and surface as an inexplicable token.
## Typed kernel-binding refusal
enum OperandUnsuitability {
ElementFormat -> bind another variant, or a kernel for this format
NonDirectView -> a view-aware kernel, or a repacked variant
MisalignedBase -> an aligned copy
Length -> reject the index; only this one is a defect
}
Four causes with four different remedies, kept apart rather than collapsed
into one message.
## Scope
Router only. Expert values still differ by kernel — incumbent Q4_K x Q8_K
integer dot against a dequantised f32 reference — and that is rung 2,
deliberately not begun here. Committing separately so that a disagreement
found during expert binding cannot require bisecting a patch containing both
systems.
larql-vindex 2272 tests, larql-compute 894, 0 clippy warnings. Acceptance run
performed with `set -euo pipefail` and an unpiped build, after an earlier
command shape was found unable to rule out executing a stale binary.
Neither roadmap mentioned VINDEX2 or VINDEX3, so three shipped milestones were invisible to anyone reading them. The programme had been tracked only in docs/vindex2-experiments.md. ROADMAP.md gains a VINDEX3 section carrying: - the thesis — VINDEX2 can observe which pages faulted, VINDEX3 can state what an operation will read before it runs; - the coexistence contract, including the standing constraint that `extract` keeps defaulting to VINDEX2 until V2-1 acceptance passes, since a silent default change would evaporate E0's premise; - the three shipped commits (f13bf38, dd2017d, f5dd256) and the rung ladder to the first VINDEX3-generated token; - a "Standing method" note recording what the programme established by repeated failure rather than by preference: bind never reconstruct, ladders rather than end-to-end tolerances, mutation-check every new test, and suspect the instrument first — this work has produced roughly three measurement defects per real code defect; - an explicit "Not discharged" list: E0-FULL's remaining rows, the OLMoE goldens that pin a since-fixed decode panic, the 15 CLI call sites still assuming VINDEX2, and why `extract --format vindex3` is deliberately not done yet. ROADMAP_STATUS.md moves the active slice from the DEC funnel to VINDEX3 execution binding, with rung 2 named as next. The DEC funnel is demoted to "previous slice" but deliberately **not** marked closed, because it is not: VINDEX3 is the storage and capability layer DEC's expert serving will bind to, and the two meet at `SelectedExpertNotResident` — the seam where a valid route with a missing local operand becomes a fetch rather than a defect. Docs only. Every link target and commit reference verified to resolve.
A VINDEX3 BoundExpert, consuming the store's own Q4_K super-blocks and a real
Gemma activation, calls the exact production expert kernel and reproduces every
observable stage of the expert path bit-for-bit.
layer 5, hidden 2816, 128 experts, top-8, intermediate 704
input: real h_post_attn, last token of "The capital of France is"
6 per-expert outputs BIT-IDENTICAL across 8 experts asserted
7 weighted reduction BIT-IDENTICAL asserted
8 block output BIT-IDENTICAL observed
oracle (relative) 1.98% of 9.687e-1 banded
Rungs 1-5 (router) remain bit-identical, unchanged from f5dd256.
Read the third column. The real Gemma layer produced a bit-identical full block
output in the acceptance run; the checkpoint-free CI test enforces exact
production-kernel parity over the padded Q4_K expert path. Rung 8 is observed,
not contracted — see "Rung 8 is reported, not asserted" below — and nothing here
promises that every future full-block invocation is bit-identical, because
scheduling can make that contract unavailable.
What this closes is the whole one-layer computation, not expert binding alone:
real activation → production router → exact top-8 and weights
→ production Q4_K × Q8_K experts → selection-order reduction
→ post-expert norm → block output
## Binding, not reimplementing
`ExpertKernel::IncumbentQ4kQ8k` calls `run_single_expert_q4k_q8k_into` over
the region's own bytes. `BoundTensor::as_blocks()` hands over the stored
super-blocks or refuses; nothing dequantises, requantises or repacks into a
kernel-shaped temporary on the way. That matters more here than at the router,
because a Q4_K bridge that dequantised and re-quantised could reach numerically
close values while proving nothing at all about block addressing.
## The oracle leg is what stops this being circular
Two calls of one function agree whatever operands they are handed, so a
bit-identical result establishes only that the handover was faithful. The same
experts are therefore also bound dequantised and run through the reference
kernel; its answer is computed independently and lands 1.98% away, which is the
low-percent figure the Q8_K activation rounding predicts.
Both legs run in CI without a checkpoint:
`the_bound_kernel_reproduces_the_incumbent_call_bit_for_bit` builds a padded
Q4_K fixture through the production quantiser and asserts bit-equality against
a direct call, and `the_two_kernels_are_not_trivially_identical` guards the
band from being vacuous.
## One binding, two kernels
Gemma's `down` is stored `[hidden, 768]` and means `[hidden, 704]`. Rung 1
resolved that with a column-prefix slice view; this rung keeps that binding and
teaches the kernel handover to read it, rather than adding a second binding of
the same bytes.
storage_cols 768 what a block-native kernel contracts over: it decodes
whole super-blocks and cannot stop inside one
role_cols 704 what the operation means
`BlockOperand` carries both, because neither alone is enough: only
`storage_cols` loses the operation's meaning, only `role_cols` misreads every
row after the first. So `as_blocks` honours a zero-start column-prefix slice —
that view is not an obstacle, it is exactly the information the kernel cannot
recover from the bytes. A row slice or a transpose is still refused, because
neither leaves a contiguous run of whole stored rows.
## Addressing is a property of the encoding
Scalar { bytes } one element per fixed slot, index-addressable
Blocked { elements, bytes } reaching one element decodes its whole block
Sizing a blocked region with a scalar stride under-counts Q4_K by ~4×, which
binds a region that looks long enough and reads past its end on the last row.
Geometry comes from `QuantFormat::packed_block_layout`, so `256 * 144` is not
spelled anywhere in vindex.
The consequence is that the refusal *moved*: a Q4_K region now binds, and it is
the per-element read that refuses. That is the right place for it — the point
of binding one is to hand its blocks to a kernel — and a codec with no
registered geometry (mxfp4, nvfp4) is still refused at bind, because a region
that cannot be sized cannot be read either.
## An activation the kernel does not implement is now refused
The incumbent's inner loop reads `match activation { GeluTanh => .., _ =>
silu(..) }`. A bank bound with `ReLU` would therefore have run SiLU, returned
finite plausible values, and signalled nothing whatsoever — the same class of
silent-wrongness as rung 1's missing per-expert scale, but with no ladder step
that would catch it, since every rung would agree.
`KernelActivationUnsupported` is a distinct variant because no rebinding of the
*bytes* repairs it: the kernel computes a different function than the recipe
specifies, and the repairs are a different kernel or a corrected recipe.
## Two more kernel-binding refusals
BlockAlignment rows (or the activation) are not whole super-blocks
-> repack, or pad the extent
Arrangement a decomposed gate/up pair where the kernel takes one slab
-> bind the fused variant, or a kernel for this arrangement
`BlockAlignment` covers the activation side too. The incumbent guards a
non-256-multiple hidden width by falling back to its f32 path; VINDEX3 refuses,
because a silent change of kernel is a silent change of answer.
Together with the shape checks these make the incumbent's own short-slab guard
— which zeroes its output and returns *successfully* — unreachable from here.
`the_bound_kernel_is_not_producing_zeros` guards the guard, since an all-zero
agreement would be two failures agreeing.
## The kernel is a bank property, and it has a session
The incumbent quantises the bank input to Q8_K once and shares it across the
bank's selected experts. That is not an optimisation to reproduce or not as
convenient — it is the shape of the call production makes, and quantising per
expert would be binding a different call. So the kernel is opened once per bank
before the expert loop, which is also why `ExpertKernel` sits on the bank rather
than on the expert: two experts of one bank on two kernels is not a binding this
runtime can express, and that restriction is correct.
## Rung 8 is reported, not asserted
`cpu_moe_forward` sums its experts through rayon or the spin pool depending on
configuration, and a tree reduction is not required to agree bit-for-bit with a
sequential one. Rung 7 therefore compares against the incumbent's own per-expert
outputs summed in *selection order*, isolating the combine from the schedule.
Rung 8 came out bit-identical on the spin-pool path — which accumulates in
selection order — and that is a measurement of one configuration, not a promise
about every one.
The oracle band is relative rather than absolute, because expert outputs are
residual-scale values whose magnitude is a property of the layer; judging them
against the router's softmax-derived 1e-3 compares the right direction on the
wrong object, and did in fact fail this run before the metric was corrected.
Its denominator is fixed and stated: `max|reference|`, never `max|found|` and
never the larger of the two. A denominator that moved with the thing being
measured would let a kernel that inflates its output report a *smaller*
percentage for a larger error, and would make two layers' figures
incomparable. Below `ORACLE_SCALE_FLOOR` the comparison switches to an absolute
one and says so, rather than dividing by a near-zero reference and emitting a
number that looks like the others but compares to nothing. Both exist so the
percentage stays meaningful when the whole-model sweep starts quoting one per
layer.
A future whole-model harness should record which of these each block comparison
is — asserted exact, observed exact, or numerically equivalent within a named
reduction-order tolerance — rather than collecting them into one "parity"
bucket, which is precisely the distinction the third column above draws.
## Test layout
`runtime/*_tests.rs` moved into a flat `runtime/tests/`, and `test_support.rs`
to `runtime/tests/support.rs`; the fixture tests likewise. Mechanical — the
runtime's own file list is what a reader scans to find the implementation, and
sixteen interleaved test files made that harder with every rung.
## Scope
One layer, one token, CPU. Not begun here: reference decoding of quantised
regions (the reference still needs a dequantised binding to serve as an oracle),
the Q6_K and Q4_0 kernels that `Addressing` now sizes but nothing binds, and
whole-model residual parity — which is the next rung, and should run as a
locked-input layer sweep first (every layer fed the incumbent's captured
`h_post_attn`, so an earlier mismatch cannot contaminate a later input) before
free propagation carries each path's own residual through the model. The first
sweep is what localises a binding defect; the second is what proves the model.
What that sweep may still find is local: a layer with a different activation or
scaling policy, a differently padded or absent expert shard, layer-specific
physical geometry, a shared rather than routed bank, alternate quantisation
metadata, or a boundary layer whose norm or reduction differs. Those are
binding defects now. The architecture is no longer the likely failure point.
larql-vindex 2044 lib tests, 0 clippy warnings, coverage policy passes at 94.05%
total with addressing.rs, expert_kernel.rs, bank.rs and region_format.rs at
100%. Acceptance run performed against the 15 GB Gemma 4 26B-A4B vindex with an
activation captured through `LARQL_CPU_DUMP_LAYERS` from a real generation.
All thirty of Gemma 4 26B-A4B's MoE layers execute through a VINDEX3-bound
route and agree with the incumbent at every checkpoint.
lyr resident top-k activation pad verdict
0 128/128 top-8 GeluTanh 704→768 observed_exact
…
29 128/128 top-8 GeluTanh 704→768 observed_exact
observed_exact 30 layer(s)
oracle, layer 5 1.98% of 9.687e-1 (fidelity, not parity)
Per layer, selection / gate weights / per-expert outputs / reduction are all
`exact` — asserted and bit-identical. The block output is `observed_exact`, so
the layer's own verdict is the weakest of the five. Reading the table: every
asserted checkpoint is a contract that held; the block figure is a measurement
of this configuration.
## Locked input, because free propagation cannot localise
Every layer is fed the incumbent's own captured `h_post_attn`. If layer 7
diverged and layer 8 then read a residual that already differed, layer 8's
mismatch would say nothing about layer 8 — the first failure would be the first
*symptom*, not the first defect. Locking the input makes each layer an
independent measurement. Composition is a separate proof and comes next.
## Classification is a library concern, not a harness one
`runtime/verdict.rs` holds the vocabulary, because three harnesses — this
sweep, the free-propagation residual comparison and the decode-parity run —
must classify the same situation the same way, and three private enums would
drift within a week.
Exact every asserted checkpoint bit-identical — a contract
ObservedExact bit-identical, uncontracted — a measurement
Equivalent within a named tolerance
NumericMismatch executed and disagreed
Refused(kind) did not execute
`ExecutionError::refusal()` sorts refusals into three kinds by *who acts on
them*: the operator who must fetch an operand, the engineer who must write a
kernel, and whoever produced an index that is wrong.
Residency the route was right, the operand lives elsewhere
Unsupported well-formed, and no bound kernel serves it
BindingDefect the binding is wrong
The match is exhaustive, so a new `ExecutionError` variant does not compile
until someone has decided which it is. `nothing_but_a_non_resident_expert_
classifies_as_residency` pins the dangerous direction specifically: `Residency`
is the one kind that reads as "nothing is wrong", so an unclassified failure
must never inherit it.
## Residency is not a parity failure, and the sweep proves it
`--shard 8` binds only the first eight experts, so every layer routes to
someone absent:
0 8/128 residency routing selected expert 24, which is not resident in
layer 0 bank 0 (this bank holds 8 of 128 experts)
…
residency 30 layer(s)
SWEEP: no layer indicts the execution.
Thirty refusals, and the run still exits zero. That is the point: the route was
correct, the expert exists in the model, and a shard that does not hold it is
behaving as designed. Counting these as mismatches would make the headline
number a measurement of slice coverage rather than of execution.
Which is also why the principal run binds the **full population** — free,
because Q4_K regions bind from the mapped bytes and nothing is materialised.
The single-layer harness binds only the selected experts, which is a legitimate
shard but the wrong default for a sweep.
## The oracle cannot ride along, and should not
The dequantised f32 leg costs ~24 MB per expert — ~3 GB per layer at full
population, ~90 GB across the sweep. It also answers a different question:
quantisation fidelity, which is a sample, where the sweep answers
implementation parity, which needs coverage. So it is `--oracle-layer N`, over
that layer's selected experts only, and it reports 1.98% relative on layer 5 —
unchanged from b2b94df, as it should be.
## A reporting defect caught before it shipped
The per-layer detail column first read "weakest at: block" for every layer,
because it flagged anything short of `Exact` and the block comparison is
unasserted by construction. That is a column that says the same thing for a
perfect run and a broken one. It now flags each checkpoint against its own
*strongest attainable* outcome — `Exact` for the four asserted ones,
`ObservedExact` for the block — and is empty when nothing is below ceiling.
## What the sweep did not find
None of the irregularities the single-layer result left open: every layer is
top-8 of 128, GeluTanh, 704→768 padded, one routed bank, same physical
geometry. Gemma is uniform. That is a fact about this model rather than a
property of the binding, and the next model in the conformance envelope is
where the classification actually earns its keep.
## Scope
One token, CPU, MoE layers only. Not begun here: free propagation (each path
carrying its own residual, which proves composition and needs the first
*differing* boundary reported separately from the earliest *causal* one), final
norm and logits, and multi-token decode parity.
larql-vindex 2057 lib tests, 0 clippy warnings, coverage policy passes at 94.05%
with verdict.rs at 100%.
Both paths start from the same embedding and each carries its own residual
through all thirty blocks. Every boundary agrees.
seam in-process via trait vs default branch BIT-IDENTICAL
boundaries 30 MoE layers observed on each path
first differing boundary none
earliest causal operation none
final free-propagated hidden BIT-IDENTICAL
The locked-input sweep (23e7621) proved every layer independently but handed
each one the incumbent's own activation, so it could not prove they compose.
This lets each path diverge if it is going to. It does not.
## The seam is now a trait, and that is a production change
`moe_ffn_block_cpu` took `Option<&RemoteMoeBackend>` — a concrete type, so a
third route meant a second optional parameter and a rule that at most one may
be `Some`. That rule is not expressible in the type, which makes it a rule
someone eventually breaks, with a layer's expert contribution coming from a
route nobody chose.
trait MoeExpertBackend one route, or the in-process default
RemoteMoeBackend experts fetched from shards
BoundMoeBackend a VINDEX3 BoundMoeOperation
InProcessMoeBackend cpu_moe_forward, made explicit
Backends derive their own operands from `weights` and a layer. The remote route
needs the router only; the bound route needs the whole layer. Preparing the
union at the call site would make the block loop know what each route reads,
which is the coupling the trait removes — and it is why the
`build_moe_router_weights` call moved out of the loop and into the remote impl.
## The seam is falsifiable, and it is checked first
`InProcessMoeBackend` must equal running with no backend at all. If it did not,
the trait would have changed the model rather than relocating a call, and every
comparison built on it would be measuring that change. The harness asserts this
before it trusts anything else, and it comes out bit-identical.
That check is the reason to route the incumbent through the trait at all — it
also gives both paths the same observable seam, so neither has to be
reconstructed to be compared.
## Two locations, never conflated
first_differing_boundary where a difference is first observable
earliest_causal_operation the first MoE that differed on identical input
Once layer 7's contribution differs, layer 8 reads a contaminated residual and
everything downstream differs — including its post-attention state, which would
otherwise read as an attention defect. The causal test is therefore
conditional: a layer indicts its own MoE only if its input residual was still
identical when it ran. Both come out `none` here, but the distinction is the
part that will matter on the first model that fails.
## Coverage: honest debt, not compliance
`moe_bound.rs` is at 65.8% and `moe_backend.rs` at 0%. Neither meets the 90%
floor, and neither is in `larql-inference`'s coverage policy scope, so nothing
gates them.
`run_layer` was split out of `forward_moe_seq` specifically to make the
execution reachable from a `MoeLayerWeights` alone — that part is tested,
including a unit-scale bit-identity check against `cpu_moe_forward`, per-position
routing, the reference pairing refusing a Q4_K store, and the unmappable-codec
tags. What remains uncovered needs a loaded `ModelWeights`, which a unit test
cannot construct cheaply. Recording that here rather than adding a baseline at
the measured value, which would gate at a floor low enough to mean nothing.
## Scope
One token, prefill only, CPU. Not begun here: final norm, logits and greedy-token
parity; multi-token decode; and the first genuinely irregular model, where the
refusal and arrangement machinery is actually exercised — Gemma is uniform
across all thirty layers, so the classification added in 23e7621 has still only
met synthetic irregularity.
larql-inference 1431 lib tests, larql-vindex 2057, workspace clippy clean,
larql-vindex coverage policy passes at 94.05%.
moe_backend.rs 0.0% → 98.91% (91/92)
moe_bound.rs 65.8% → 96.95% (318/328)
Both are now in `larql-inference`'s coverage policy, so they are gated at the
90% default rather than sitting outside its scope.
## The 65.8% was partly a measurement artefact
`moe_bound.rs` reported 65.8%, then 72.6% after tests that plainly exercised
the uncovered code. The missing-lines list pointed at doc comments, which
cannot be uncovered — the profdata was stale, and `cargo fmt` had shifted every
line under it. A clean `cargo llvm-cov clean --workspace` moved the same code
from 318/438 to 318/328: identical covered count, 110 phantom lines gone.
Worth recording because the failure mode is quiet. A stale profile does not
error; it reports a plausible number against the wrong line map, and the
natural response is to write tests for code that is already covered. The
guard is the one that caught it here: if the uncovered list names a line that
cannot execute, the measurement is wrong before the code is.
## What the tests actually pin
`InProcessMoeBackend` is byte-identical to the block loop's default branch,
per position, against `cpu_moe_forward` on the synthetic Gemma fixture — the
same property the free-propagation harness checks against the real model, now
also checked without one.
`BoundMoeBackend` over a loaded `ModelWeights`: the wrapper resolves the layer,
binds its whole population and runs every position, agreeing with the
in-process route within a summation-order band. The fixture stores BF16
experts, which makes it the useful case rather than a weaker one — the
reference kernel decodes them, and the production Q4_K pairing must *refuse*
them. That refusal is asserted to arrive as `RefusalKind::Unsupported`, so a
wrapped error still carries the classification `verdict.rs` added.
Both routes contribute zeros for a layer with no expert weights. The block loop
adds the contribution unconditionally, so a backend that errored there would
change the model rather than the route.
## Not closed
`moe_remote/backend.rs` sits at 10.8%, unchanged and pre-existing — this commit
only appended a trait impl to it. It is not in the policy's scope and this
commit does not add it, because bringing it to the floor is a piece of work
about the remote path rather than about VINDEX3.
larql-inference 1439 lib tests, clippy clean on the touched files, both coverage
policies pass.
The free-propagation harness now runs to the model's own output.
pre-final-norm hidden BIT-IDENTICAL
post-final-norm hidden BIT-IDENTICAL
top-k token ids IDENTICAL
top-k scores BIT-IDENTICAL
argmax token " capital" vs " capital"
argmax margin 9.483586e-1 vs 9.483586e-1
With the composition rungs above it unchanged: the seam is bit-identical to the
default branch, no boundary differs across 30 MoE layers, and no MoE differed on
an identical input.
## Production functions, not a local endpoint
`apply_norm` is the one `logits_to_predictions` itself calls, and the predictor
is the one generation calls. A harness that rolled its own final norm or lm_head
would be comparing the harness — the same reason the MoE comparison binds
`larql-compute`'s kernels rather than reimplementing them.
## The margin is recorded even though it is implied
Identical scores make an identical margin a tautology here. It is reported
anyway because the approximate and partially-resident runs that come next need
this exact report shape, and a field that only appears once something disagrees
is a field nobody can compare against.
## What the token is and is not
The harness encodes the raw prompt, where the CLI applies the chat template — so
the argmax is " capital" rather than the " Paris" a templated run produces. That
is a property of the prompt this harness feeds, not of either execution path.
Both paths receive byte-identical input, which is the only thing the comparison
claims. The token is a parity observable, not a quality claim.
## Scope
One token, prefill only, CPU. Not begun here: multi-token decode with KV reuse,
which is what "VINDEX3 runs Gemma normally" would require, and selective page
residency.
Also not done, and named rather than half-built: the stale-profile guard. The
65.8%-that-was-really-96.9% in 0465978 came from a coverage profile whose line
map predated a reformat. CI is immune — it runs on a fresh checkout — so this is
a local-development hazard, and the summary-only policy script cannot see the
signature that gives it away. Worth solving deliberately rather than by bolting
a `clean` onto a command that does not currently need one.
`MoeFfn { weights, moe: &dyn MoeExpertBackend }` — the `FfnBackend` a KvEngine
drives per MoE layer, over whichever expert route is installed.
## The seam stopped one level short
The composition rung made the *block loop's* seam a trait, and proved a VINDEX3
bound route composes through 30 layers to a bit-identical token. But that ran on
the full-recompute path. The engine's adapter — `RemoteMoeFfn`, which is what
`generate_with_engine_resident` actually drives — still named one concrete
backend in its type:
pub struct RemoteMoeFfn<'a> {
pub weights: &'a ModelWeights,
pub remote: &'a RemoteMoeBackend, // <- concrete
}
So the bound route was reachable from prefill and not from decode. Nothing was
wrong; the generalisation simply had not been carried up.
`RemoteMoeFfn` now delegates rather than being replaced. It keeps its
`{ weights, remote }` literal — load-bearing at a CLI call site and in two
roadmaps — and keeps reporting `"remote-moe"`, while `MoeFfn` reports whichever
route it carries.
## What the tests pin
`the_remote_adapter_still_equals_the_general_one` is the same property the block
loop's seam had to prove: same weights, same input, same output, so the
delegation relocates the call and nothing else. The name is the one deliberate
difference.
`a_bound_route_can_drive_the_engine_adapter` is the point of the change — a
VINDEX3 route producing a finite, non-zero contribution through the engine's
FFN seam, which the remote-typed adapter made impossible to express.
`a_refusing_route_still_returns_the_dense_half` pins the failure shape.
`moe_ffn_block_cpu` logs and leaves `h2` zero when a route refuses, so a refused
expert contribution is *not* the same output as one that ran. Without this a
refusal would look like a plausible layer rather than a missing one.
## Not done, and deliberately not half-done
Multi-token decode parity itself. The harness needs resident f32 dequantisation
of every layer, a built `EngineKind`, `generate_with_engine_resident`, and
`larql-kv` as a dev-dependency of the harness crate. That is a known shape
rather than an unknown one — the CLI does exactly this at
`run_cmd.rs`'s engine branch — but it is more than a careful hour, and a
half-driven engine would produce a comparison nobody should trust.
This commit removes the blocker; the run is the next rung.
`ffn_adapter.rs` is at 82.3%, below the 90% floor and not in the policy's
scope. The uncovered remainder is the pre-existing dense-fallback surface, not
the new adapter. Flagging rather than gating at a floor that would lock in the
gap.
larql-inference 1442 lib tests, clippy clean on the touched file.
Tests whether the real routed expert set is necessary and sufficient
for identical MoE execution, using the runtime's existing
ExecutionError::SelectedExpertNotResident mechanism against a real
Gemma 4 26B-A4B MoE layer and real captured routing decisions (not the
4-expert synthetic fixture in runtime/tests/placement.rs).
Four-rung ladder, all passing across 5 layers spanning early/mid/late
depth (0, 5, 10, 20, 29):
1. full (128) vs minimal (8 routed) population -> bit-identical (sufficiency)
2. remove one routed expert -> SelectedExpertNotResident, correct
{expert, resident, population} metadata, no fallback (necessity)
3. same removed expert, padded back to full byte capacity with an
unselected substitute -> still refused (capacity is not residency)
4. exact missing expert restored -> exact parity restored
Reframes MAP-3's original design: literal OS-level page eviction can't
cause a catchable execution failure (mmap transparently re-faults from
disk), so this tests the real, working analogue -- logical operand
presence in the bound plan, independent of physical location.
Registered as chuk-experiments map-3a (confirmed).
…pproximation
MoeFfn::best_effort(..) log, contribute zeros, return the dense half
MoeFfn::strict(..) record the refusal; the caller must consult it
## The behaviour this constrains
`moe_ffn_block_cpu` logs a route's error, leaves the expert contribution at
zero and returns the dense half. For best-effort remote inference that is
correct — a degraded continuation beats no continuation when a shard is
unreachable. For a correctness experiment it is the exact failure to avoid:
SelectedExpertNotResident
→ swallowed and logged
→ dense half returned
→ plausible, numerically wrong continuation
A selected-but-absent expert must not silently become an FFN-skipping
approximation.
## Taken before the decode run rather than after
The ladder had this third, behind teacher-forced and free-running decode. It
moves first because it is a prerequisite for the decode rung's *diagnosis*, not
only for MAP-3A: a swallowed refusal reaches the comparison as a numeric
difference, so the divergence report would name a `NumericMismatch` where the
truth is `Residency`. That is precisely the misdiagnosis `runtime/verdict.rs`
was written to prevent, reappearing one layer up.
## Caught on the way in, not on the way out
`FfnBackend::forward_moe_full_layer` returns `Option<Array2<f32>>` — no error
channel — and widening it would reach every engine. So the refusal is
intercepted *before* `moe_ffn_block_cpu` handles it: `MoeFfn` wraps its route in
a `RefusalRecorder` that implements `MoeExpertBackend`, records, and passes the
error through unchanged. No signature moves and no engine changes.
Strictness lives in the constructor rather than a settable field, and the check
is `refusal()` / `all_experts_executed()` rather than a log line someone has to
read. The recorded refusal keeps its `RefusalKind` rather than a flattened
string, because a gate that could not tell `Residency` from `Unsupported` could
not tell a sharded deployment from a broken one.
Only the first refusal is kept. Later layers usually repeat the same cause, so
the earliest is the diagnosis — the same first-versus-causal distinction the
divergence report makes.
## What strict does *not* yet do
It does not stop the block. `FfnBackend` cannot say "no answer", so a strict
adapter still returns the dense half and relies on the caller consulting
`refusal()` before accepting logits. That is a contract, not a type-level
guarantee, and it is the weaker of the two. Making it unrepresentable means
giving `forward_moe_full_layer` an error channel, which is an engine-wide
change and deserves its own rung rather than riding on this one.
`RemoteMoeFfn` stays best-effort, pinned by a test: its callers depend on
degrading rather than stopping.
## Coverage
`ffn_adapter.rs` 82.3% → 88.2%. Still under the floor and still not gated; the
remainder is the pre-existing dense-fallback surface rather than anything added
here.
larql-inference 1447 lib tests, clippy clean on the touched file.
The real `KvEngine` decode loop, two expert routes, one model.
== stage 1: teacher-forced ==
prefill exact
step 1 fed " capital" exact | argmax " of" ==
step 2 fed " of" exact | argmax " France" ==
step 3 fed " France" exact | argmax " is" ==
step 4 fed " is" exact | argmax " capital"==
step 5 fed " capital" exact | argmax " of" ==
step 6 fed " of" exact | argmax " France" ==
first differing boundary none
earliest causal step none
== stage 2: free-running greedy ==
6/6 steps identical token and identical margin
in-process capital of France is capital of
bound capital of France is capital of
This is the condition the bound route had never met. Every prior rung ran on the
full-recompute prefill path, one token, no KV reuse — and `BoundMoeBackend` binds
per call against weights borrowed from the map, which decode is the thing that
stresses.
## Teacher-forced first, because free-running cannot localise
If step 3's hidden state differs and step 4 is then fed a different token,
step 4's difference says nothing about step 4 — the comparison has branched and
every later number is about the branch. Feeding both engines an identical token
sequence keeps each step an independent measurement, which is what the
locked-input sweep did for layers, one dimension over.
Under teacher-forcing the in-process route chooses and both engines are fed its
choice. The bound route's own preference is computed and compared but never
acted on, so a divergence is recorded without being allowed to propagate.
Stage 2 runs only because stage 1 passed, and it is what moves the claim from
"equivalent under controlled input" to "generates the same text".
## Refusal is checked before any number
Both routes are driven through `MoeFfn::strict`, and `classify` consults
`refusal()` *before* computing a difference. Without that ordering a missing
operand would arrive as a numeric mismatch and be reported as a wrong answer —
the misdiagnosis `runtime/verdict.rs` exists to prevent, and the reason
2067f5b was taken ahead of this rung rather than after it.
A step that refused is classified and stops the comparison. Only a step where
both routes executed makes a numeric verdict meaningful.
## What the tokens are and are not
The harness encodes the raw prompt where the CLI applies a chat template, so the
continuation loops — " capital of France is capital of". Both paths produce it
identically, which is the only thing the comparison claims. The margins are
0.948 to 0.9999, so no step sat near a tie: this is parity decided by the
scores, not by tie handling.
## Scope
Greedy only, CPU, `EngineKind::Standard`, one prompt. Not covered: sampling,
the windowed and boundary-KV engines, long contexts where the cache evicts, and
Metal. Nor the *lifecycle* question — the bound plan is rebuilt per call, and
whether retaining it per layer is the production shape is a performance question
this rung deliberately does not answer, having proved correctness first.
`larql-kv` is now a dev-dependency of `larql-vindex`. Dev-only: nothing in the
library depends on the engine layer.
larql-vindex 2057 lib tests, clippy clean on the new harness.
`larql-execution` — `RefusalKind`, `ExecutionRefusal`, `BoxRefusal`. Zero
larql-* dependencies, zero dependencies at all.
## A dependency cycle pointed at a missing layer
Widening `FfnBackend::forward_moe_full_layer` to a `Result` needs the error to
carry its classification. It could not:
FfnBackend in larql-compute (deps: larql-models)
RefusalKind in larql-vindex (deps: larql-compute, ...)
The arrow points the wrong way, and larql-compute cannot gain a dependency on
larql-vindex without a cycle. So the vocabulary belonged to neither crate — it
is a cross-runtime execution contract, and the cycle was the diagnosis rather
than an obstacle to route around.
The alternatives were worse. A boundary trait returning `kind() -> &'static str`
solves Cargo and reintroduces the semantics problem: "residency",
"not-resident", "missing-expert" all become expressible, and exhaustive matching
is gone. Moving it into `larql-models` would put runtime semantics in a crate
about model structure. Leaving it audited keeps invalid output representable.
Not speculative extraction: `larql-vindex-spec` set the precedent — a
dependency-light contract crate with four consumers — and this has three on day
one.
## The axis is the response, not the cause
Three variants because there are three distinct responses, each with a different
person on the other end:
Residency same operation, operand becomes available -> may succeed
Unsupported same operands, another capable executor -> may succeed
BindingDefect the binding or artifact itself must change
An earlier proposal split `Unsupported` into `DecomposedProjection`,
`Activation` and `Arrangement` and dropped `BindingDefect`. Both halves of that
are wrong. Those three are *causes of* unsupported-ness — they already live in
`OperandUnsuitability`, with their own detail — and promoting them puts two
abstraction levels in one enum, leaving a caller unable to switch on the thing
it must act on. And `BindingDefect` is the only kind meaning *reject the
artifact*: `ExpertOutOfRange`, `ShortRegion` and inconsistent plans are fixed by
neither fetching an operand nor choosing another kernel.
The invariant, documented on the type so a future variant has a test to meet:
> `BindingDefect` means repeating the operation with more residency, or
> through any other capable executor, cannot make *this* bound plan valid.
`the_two_predicates_partition_the_vocabulary` pins it — every kind must be on
exactly one side of `indicts_the_artifact` / `is_recoverable_without_rebinding`,
so a variant that is neither or both fails.
## Transport failures are deliberately not a fourth kind
A timeout, a refused connection, an expired lease are execution-*attempt*
failures, often retryable, and they belong to whatever transport owns them. One
may *conclude* in `Residency` when the semantic finding is that the operand is
unavailable to this plan. Admitting them directly would turn the enum from
execution semantics into a catalogue of operational failures.
## Concrete errors stay where they belong
`ExecutionError` implements the trait by delegating to its existing `refusal()`
rather than re-matching — two exhaustive matches over the same variants is
exactly how a classification drifts. `runtime::RefusalKind` still resolves,
re-exported, so no call site moves.
`larql-execution` 100% line coverage, 6 tests; larql-vindex 2057, unchanged.
The `FfnBackend` widening this unblocks is the next rung.
`ExecutionRefusal::kind` delegates to `ExecutionError::refusal` precisely so the two cannot disagree — but a delegation nothing calls is a delegation nobody has checked, and extracting the vocabulary in f5c6637 dropped `error.rs` from 100% to 66.67% for exactly that reason. Caught by the coverage gate, not by review. Two tests. The first walks every `ExecutionError` variant through both paths and asserts they agree. The second boxes one as `BoxRefusal` and checks that both levels survive the crossing the trait exists for — the response category (`Residency`, not indicting the artifact) *and* the concrete diagnosis (expert 90, 8 of 128), rather than the detail being flattened into the category. larql-vindex coverage policy back to passing at 94.05%.
fn forward_moe_full_layer(..) -> Result<Option<Array2<f32>>, BoxRefusal> Six implementations, four call sites, three crates. The classification now crosses the boundary that most needed it — the one between a route that refused and an engine deciding what to do about it — which is what extracting `larql-execution` in f5c6637 was for. ## Three outcomes, never two Ok(Some(h_out)) the layer executed Ok(None) this backend does not serve this layer; fall back Err(refusal) a required operation could not execute `Ok(None)` must never report a refusal. It means *not applicable* — the caller runs its own dispatch and the result is still correct. `Err` means a routed operation was required and did not happen, so anything the caller assembles is incomplete. Conflating them puts the caller back where the error channel was added to rescue it from, and the trait doc says so. Pinned at the two places the distinction is easiest to lose: a sharded backend with no shard for a layer, and a v0 backend with no MoE hook, both assert `Ok(None)` rather than merely "not an error". ## The backend reports; the caller decides `MoeFfn` previously chose *how failure was communicated* — best-effort swallowed, strict recorded for later inspection. Now it reports, and policy sits at the call site: Strict Err propagates; the dense half is computed and discarded BestEffort logs the degradation explicitly, returns the dense half The discard is deliberate. `moe_ffn_block_cpu` has already logged and zeroed by the time the recorder is consulted, so `out` is a dense half wearing the shape of an answer; under `Strict` it must not escape. That costs one wasted dense pass on an error path and buys the guarantee no caller can consume a partial layer — the difference between this and the audited contract it replaces, where the caller had to remember to ask. `RecordedRefusal` is now an error in its own right, so it crosses as a `BoxRefusal` carrying both levels: the category an engine switches on and the concrete message the route produced. ## Transport failures were not promoted The HTTP walk backend returned `None` on every transport failure, meaning "fall back". It still does. A timeout or refused connection is an execution-*attempt* failure, often retryable, and it is not that layer's place to decide it has become a semantic refusal. Promoting them under cover of a signature migration would have changed the remote walk path's behaviour silently. ## Where propagation still stops `ffn_or_moe_layer` (both copies) and the remote-FFN recompute loop return bare arrays, so a refusal is *named* — with its `RefusalKind` — and the caller falls back. A strict route reaching those paths still degrades like a best-effort one. Widening them into the engine step is the next ring; the comment at each site says so rather than leaving it to be discovered. So this rung delivers the channel and the policy split, not yet the end-to-end guarantee. ## Verification Full Gemma ladder re-run after the migration, to show that widening the error channel changed no successful computation: layer sweep no layer indicts the execution free propagation composition and predictions agree MAP-3A CONFIRMED, 4/4 rungs decode parity identical teacher-forced and free-running larql run "Paris" Workspace 10165 tests, 0 failures, 0 clippy errors. Both coverage policies pass.
The FfnBackend ring (24a267c) gave a refusal a typed channel but stopped one level short: `ffn_or_moe_layer` returned a bare array, so a strict route reaching it degraded exactly like a best-effort one. This widens the six dispatch helpers and terminates the channel at the engine. ## Three outcomes, all the way out DispatchOutcome<T> = Result<Option<T>, BoxRefusal> Ok(Some(_)) the dispatch produced a complete result Ok(None) nothing to do, or the backend declined this shape Err(refusal) a routed operation was required and did not execute `Ok(None)` carries exactly what a bare `None` used to, so a declining backend still becomes `BackendFailure`. `Err` is the channel that did not exist. ## Transactional decode A decode step mutates before it can know whether it will finish: each layer's attention appends the token's K/V, and only then can the FFN refuse. So a step that does not complete rewinds every handle to its entry length, via a new `KvDispatch::truncate_kv` — the inverse of an append, where `clip_kv` keeps the tail this keeps the head. Its default answers `false` rather than panicking, because "this backend cannot rewind" is a state to handle, not a bug. Windowed caches are the subtle case: append-then-evict leaves the row *count* unchanged while the oldest row is gone, so length cannot detect it. `rewind_is_sound` tests whether every layer had room before the step began. Where the rewind cannot be trusted the engine says so — `StateInvalidated` wraps the cause, the engine refuses further decode, and `prefill` clears it. ## The API said two incompatible things `is_recoverable()` answered "could this operation succeed?" while callers read it as "can I retry?". A Residency refusal that invalidated the cache is recoverable in the first sense and catastrophic in the second. Now: operation_is_recoverable() could this operation ever succeed? engine_state_is_retryable() is this engine instance still usable? is_recoverable() both — what a harness may act on `EngineError` loses Clone/PartialEq/Eq: it carries the refusal itself so the classification and the concrete error reach a handler together. `ScoreOutcome` gains three variants rather than one. A BindingDefect is not a coverage deficit, and a dead engine is not a gap in a run. ## Scope, stated honestly This covers `StandardEngine`. The five engines routing through larql-kv's `layer_ffn_or_moe` still swallow refusals; widening them is the next ring and the gate suite's module doc names them. ## Verification 24 gate cases (8 entry points x 3 RefusalKind) assert Err with the original kind and no output; a retry after a rewound refusal is bit-identical to never having refused. Gemma ladder 10/10. Coverage policy passes at 96.06%. `kv_engine.rs` (1300 lines) split into kv_engine/{error,info,stages,kv, retrieval,any,test_stubs}.rs — structure only, largest file now 343.
Every VINDEX3 parity result so far bound its operands out of a VINDEX2 file —
`index.json.version` was 2 at every step, on every model. What was proven was
the runtime:
proven the VINDEX3 executor, fed VINDEX2 operands, matches production
not proven a VINDEX3 container can be written, opened, bound and executed
Nothing could write one. `ContainerGeneration::V3` appeared in exactly one
test, constructed from a hand-written JSON string.
## The two ends
The middle of the stack was already built — LYRW v2 tables, the programme
manifest, capability and authority, the bound executor. What was missing was
the two ends, and an end nothing exercises is an end nobody has debugged.
<root>/
├── index.json schema 3 — sole root authority (§12)
├── moe_manifest.json programme description (§8)
└── routed/layer_000.lyrw LYRW v2 bank (§6)
No transcoding in the assembler: a container writer that also converted formats
would be the silent conversion §9.1 forbids, one layer below where anyone would
look for it.
## Fixture A, and what it proves
Two arms execute identical arithmetic on identical weights; only the source
differs — in-memory buffer vs a container resolved through index.json v3 →
moe_manifest.json → LYRW v2 region table. Output is bit-identical.
Storage fidelity and semantic fidelity are asserted separately, so a failure
localises: one test says the container stored the right bytes, the other says
the binding read them right. A single assertion cannot distinguish those.
Fused and decomposed FC1 storage agree under one programme id. `gated-mlp-v1`
declares both legal, so the manifest identifies *semantics* while the region
layout supplies one valid physical implementation — the separation K3 needs
when packing, quantisation, banking and transforms all vary underneath a fixed
logical contract.
Expert count and top-K come from the container, not a constant: (8,2), (32,4)
and (5,1) run through one code path. The shared-bank axis of that row stays
open — `BoundMoeOperation::banks` documents unrouted shared banks as arriving
with the Mini-K3 rung, so there is nothing to test against yet.
## verify means something for a VINDEX3 container
A VINDEX2 container is opaque blobs, so verifying it means checksums. A
VINDEX3 container declares its own structure, so it can answer *will this
bind?* without executing: index parses, manifest validates, storage keys
resolve, segments parse, roles satisfy the programme, regions are in bounds.
Defects carry {layer, bank, role, segment}. Execution parity is deliberately
excluded — it would make routine verification cost a forward pass.
## CLI
`show` and `verify` dispatch on `detect_generation` and let each generation
describe itself. VINDEX3 is not normalised into a VINDEX2-shaped summary:
programme id, storage key, manifest validity and bindability have no VINDEX2
equivalent and are what a binding failure is diagnosed from.
Gate status: V2-0 and V2-1 close for the rows fixture A can carry. Variant
selection refusal, shared banks and WALK/DESCRIBE parity remain open, so
`extract` still writes VINDEX2 and the ABI is not frozen.
`vindex2-format-spec.md` was titled VINDEX3 and versioned 2.0-draft-2, which
put the same off-by-one between name and discriminator that `generation.rs`
already records having fixed once in code.
index.json.version 1,2 → VINDEX2 index.json.version 3 → VINDEX3
Renamed to vindex3-format-spec.md / vindex3-experiments.md, version header to
3.0-draft-2, and two stale generation references updated (the adjacent bullet
already said "requires VINDEX3 loader").
Three things keep a 2 on purpose and now say so in the spec: the pre-registered
`V2-x` gate ids, the `vindex2` registry programme — both external keys with
results recorded against them — and `lyrw2`/`FORMAT_VERSION = 2`, which is the
bank format's own version on a different axis from the container's.
Records the container result and per-row gate status in the experiments
programme, adds the refusal/retry contract to larql-kv's state policy, and
notes container generations in the README where the vindex layout is described.
Also clears the larql-cli clippy backlog its CI comment estimated at ~82 errors
(actually 6) plus 3 in examples, so `clippy --workspace --all-targets -D
warnings` is now clean. Two unconstructed enum rungs are kept with a stated
reason rather than deleted: they are modelled vocabulary, and deleting an
unconstructed rung is how a ladder collapses back into the coarse form that
caused the wrong MXFP4 claim.
Edits made after the last fmt pass; caught by larql-vindex's format gate.
`latent_mask.rs` sat at 66.43% against the crate's 90% floor — the pure selection logic was well covered, the accumulation and dump paths were not. CI's coverage gate caught it; it predates this branch's other work. Four tests for `record_stats`/`dump_stats`: the no-path early return, a survival count reaching the dumped file, a short row growing to fit a wider mask without losing what it had already accumulated, and the sampled costats matrix needing its own switch. `STATS`/`COSTATS` are process-global while `set_env_override` is thread-local, so these assert structure — the file exists, parses, and carries our layer with a non-zero count — rather than exact totals, which a parallel test's accumulation would perturb.
Five items, ordered by what unblocks what, each with the condition that closes it so "done" is not a judgement call. The first is the one that constrains the baseline tag: strict refusal is real for StandardEngine only, and the five engines routing through larql-kv's `layer_ffn_or_moe` still degrade. Records that the hard half is not the ten call sites but the per-engine rewind-or-invalidate decision — each engine holds its own continuation state, so what can be undone differs by what it treats as canonical. Also records the continuation-state intervention harness as adjacent work, with the two things PR #197 established that it should inherit rather than rediscover: capability-on-the-trait with an unsupported default (truncate_kv), and the sliding-window trap where a length-based checkpoint is not a checkpoint.
This was referenced Aug 2, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two things land here. They were built in one tree and verified together, so
this is the combined candidate rather than two branches merged.
1. Strict refusal survives dispatch and the engine boundary
The
FfnBackendring gave a refusal a typed channel but stopped one levelshort —
ffn_or_moe_layerreturned a bare array, so a strict route degradedexactly like a best-effort one. The six
kv_*_via_dispatchhelpers now carrythree outcomes all the way out:
Decode is transactional. Attention appends the token's K/V before the FFN
can refuse, so a failed step rewinds every handle via a new
KvDispatch::truncate_kv— the inverse of an append (clip_kvkeeps the tail,this keeps the head). Windowed caches are the subtle case: append-then-evict
leaves the row count unchanged while the oldest row is gone, so
rewind_is_soundasks whether every layer had room, not whether the count cameback. Where the rewind cannot be trusted,
StateInvalidatedwraps the causeand the engine refuses further decode until re-prefilled.
The API said two incompatible things.
is_recoverable()answered "couldthis operation succeed?" while callers read it as "can I retry?" — and a
Residencyrefusal that invalidated the cache is recoverable in the firstsense and catastrophic in the second. Now split:
2. VINDEX3 becomes an executable disk container
Every VINDEX3 parity result before this bound its operands out of a VINDEX2
file. What was proven was the runtime, not the format — nothing could write a
VINDEX3 container, and
ContainerGeneration::V3appeared only in a detectiontest built from a JSON string.
Conformance fixture A now round-trips end to end — write → detect → open →
validate → resolve storage key → resolve LYRW v2 regions → bind → execute —
bit-identically to the same weights held in memory. Fused and decomposed
FC1 storage agree under one programme id, and expert count / top-K come from
the container rather than a constant.
larql showandlarql verifydispatch ondetect_generation; VINDEX3 is notnormalised into a VINDEX2-shaped summary. For a VINDEX3 container
verifymeans will this bind? — index parses, manifest validates, keys resolve,
segments parse, roles satisfy the programme, regions in bounds — with defects
carrying
{layer, bank, role, segment}.Verification
--workspace --all-targets -D warningsScope, stated honestly
StandardEngine. The five engines routing throughlarql-kv's
layer_ffn_or_moestill swallow; that is the next ring and thegate suite's module doc names them.
extractstill writesVINDEX2 and the ABI is not frozen.
shared banks (blocked on the Mini-K3 rung) and WALK/DESCRIBE parity remain
open, and are recorded as such in the experiments programme.
Accurate summary: Gemma-conformant architecture, executable VINDEX3 container
proven on fixture A, prepared for the K3 proof ladder.