REPRESENT codec contract: the representation/execution trait, extracted from the five stored encodings - #419
Merged
Conversation
…m the encodings the container carries
The facts a stored encoding must declare before LARQL plans, binds or
executes it lived in seven places: the K-quant geometry table, the
expert-encoding match, the NVFP4 pack layout, private group constants
in the loader, the runtime's residency-format enum, the compute crate's
`QuantMatVec` dispatch and the K3 ledger's maturity ladder. Two formats
had already been routed around the trait meant to serve them because a
single `&[u8]` had nowhere to put their scales.
`RepresentationCodec` (represent/codec/) is that contract, extracted
from what the container already holds rather than designed against a
hypothesis: float tensor tables, Q4_K/Q6_K/Q8_0, NVFP4, MXFP4, and the
LYRW v2 banks that pair their scales. It declares six things:
identity ABI family + revision, so the file names its contract
independently of the implementation registered
streams a NAMED SET plus auxiliary operands (empty today, so a
codebook arrives as a dependency, not a stream)
capabilities access granularity, grouping, alignment, refused by
name in preflight
extents one certificate per depth; every codec answers 0
decode mandatory, range-aware, to canonical f32
realizations residency declared per REALIZATION: the decode path and
each direct kernel carry their own profile, so a
fallback is a different realization with a different
declared cost, never a quiet substitution
MXFP4 is the multi-stream witness: it takes the default `bind_packed`
and is refused by name when handed one payload, the answer
`QuantMatVec` could only give as `None`. NVFP4 overrides it with the
split its shape derives. Nothing in any of the five is privileged, and
a foreign stub codec registers and gets the trait's defaults.
Three consumers now derive from the codec instead of restating its
facts: `OperandStore::load` dispatches every stored dtype through the
registry (an unregistered dtype is refused naming the registered ones),
`CodecIdentity::admit` is the registry's gate, and
`ExpertEncoding::matrix_bytes` prices a bank through the codec. The
loader's MXFP4 group constants and the graph builder's MXFP4 label each
have one home. `decode_nvfp4_operand` is gone; its test calls the codec.
Loading is stricter in two places, and one of them found something. A
K-quant row must be a whole number of blocks at read time, as the
compiler always required at write time. A float operand shorter than
its declared shape is refused instead of decoded short, which showed
the routed-FFN loader reading the router before checking the declared
bank geometry; it now checks geometry first, as its own test had always
claimed and only held because the widener under-decoded in silence.
Two tests migrated because `mxfp4` stopped being an alien family.
Stated boundaries: the executor still selects plans itself and emits no
requested/selected/reason trace; only CPU realizations are declared;
K-quants have no direct realization on the V3 CPU path. The programme
this opens, entropy-coded bf16 as the hostile sixth codec, then
progressive, then VQ, is in docs/represent-codec-contract.md.
chrishayuk
added a commit
that referenced
this pull request
Sep 5, 2026
…admits it
PARETO-1 rung A decodes every K-quant to f32 and runs BLAS over the
image: correct, and 97.4 GB of widened weight traffic per token on
Qwen3.8-27B where the artifact holds 24.4. v2 made that survivable
(file-backed staging) and then I/O-bound. This is v3: the stored blocks
reach the kernel as the artifact's own bytes, so the representation
VINDEX3 stores is the representation LARQL executes.
LoadedWeight::KQuant {blocks, codec}
-> WeightSlice::KQuant
-> WeightRows::KQuant
-> PhysicalProjectionPlan::FusedKQuant
-> KQuant::gemv
Four decisions, each stated where it lives:
- The codec owns the kernel association (`KQuant::gemv`, beside
encode/decode), so one place says which layout's kernel a name's
bytes reach. `larql-compute` still carries a `QuantFormat::Q8_0` that
means int8 codes with an EXTERNAL f32 scale stream; ggml's Q8_0 never
reaches it, and a test says so.
- The K-quant slice is exact length, never a prefix cut. Q6_K bytes
labelled Q4_K are LONGER than Q4_K wants: they pass a prefix cut and
get walked at 144 bytes over 210-byte blocks. The other arms tolerate
a longer slice because AlignedBytes pads to the page; a K-quant is
plain bytes bound at load against the codec's own plan of the shape.
- `FusedKQuant` is LibraryOwned: compute's Q4_K/Q6_K kernels thread
themselves and the new Q8_0 gemv was written to match, so the
executor calls it once rather than nesting a fan-out.
- `LARQL_KQUANT_EXEC=widen` restores v2's path in the same binary, and
only the exact word widens. The executor prints `projection plans:`
from its ledger so the arm that RAN is observed, and `run_bank.py`
refuses an arm that did not run as named. `runtime compile: 0` is
identical in both arms and cannot tell them apart.
The gate was frozen before the wiring existed (top-1 flips 0, mean KL
<= 5.2e-6, max KL <= 1.73e-5 bits/token: a tenth of a nuisance already
refused on this model) and the thresholds never moved. Three layers,
all against decode-then-multiply on the SAME stored bytes:
Layer 1, kernel vs decoder on llama.cpp's bytes: Q8_0 bit-for-bit,
Q6_K/Q4_K <= 8.6e-7 relative; it now calls the production association,
so it covers the dispatch table too.
Layer 2, a real projection through the executor on the real anchors:
Q8_0 direct is BIT-EXACT against the scalar decode-then-multiply on
[17408,5120], and its whole disagreement with BLAS (1.94e-7 of row
scale) is BLAS's own; Q6_K/Q4_K sit at 2e-8..2e-7, the same order as
BLAS-vs-scalar. Six mutations fail loudly: not a pack, three wrong
geometries, wrong codec in both directions, a perturbed byte. The
elementwise relative metric read 6% on the bit-exact kernel and is
printed, not thresholded: accumulation-order error is bounded by the
row's sum|w x|, not by the output.
Layer 3, end-to-end on the seven-prompt, 223-position set declared
before any v3 logit existed:
anchor top-1 flips mean KL max KL (gate 0 / 5.2e-6 / 1.73e-5)
U8 0 1.418e-11 1.205e-10
U6 0 2.361e-12 2.106e-11
U4 0 2.210e-12 8.465e-11
Five orders of magnitude under the gate on every anchor. v3-widen is
byte-identical to the frozen v2 executor's U8 dumps on all six prompts
it had produced, and v3-direct differs from them on all six, so the
wiring did not disturb the f32 path and the comparison can tell the
arms apart. Per position the decoder streams 25.9 / 20.0 / 13.7 GB
direct against 97.4 widened. Throughput is not the gate and is not
uniform: U4 direct runs 2.0x faster than widen, U6 1.8x slower, U8
2.8x slower — the Q8_0 gemv is the exactness-first scalar loop, and a
SIMD one is a throughput rung that must be re-qualified.
Found on the way: the projection ledger's `all()` and `reset()` omitted
FusedNvfp4, so a decode over a compiled NVFP4 pack tallied bytes nowhere
and no reset cleared them. Both now enumerate every plan.
Coverage made two more things testable rather than exempt. The Q4 class
set and the arithmetic arm are resolved by pure `from_env_value`
functions behind their `OnceLock` reads, like the K-quant arm, so both
answers of each have tests. The staging arena's open-and-unlink is its
own function, so the refusal that names the remedy is exercised without
depending on which test staged an image first.
Rebased onto main (25 commits, incl. #419's representation/execution
contract in `represent/codec/`). Integrated there rather than beside it:
the K-quant codec now declares its direct realization —
`Acceleration::cpu(FusedKQuant, ResidencyProfile::stored(bpw))` — so the
contract that main added lists Q4_K/Q6_K/Q8_0 among the codecs with a CPU
realization instead of among those that decode to f32. One resident
`WeightFormat::KQuant` serves all three; the codec identity rides in the
bound operand, not the format, which is the one place the contract test's
format-to-codec map needed widening. The real-anchor equivalence numbers
are byte-identical across the rebase (Q8_0 still bit-exact vs scalar),
so main's contract change moved no logit.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Rung 1 of the REPRESENT codec-contract programme:
RepresentationCodec, extracted from the five encodings the container already carries, with the registry in front of it and three consumers rewired to it. Design and programme:docs/represent-codec-contract.md.What it declares
CodecIdentity), so the file names its contract independently of the registered implementationAuxiliaryOperands(empty today; a codebook arrives as a dependency, not a stream)Extracted, not designed
RegionFormat→ codec,Packing→ single / paired binding).bind_packedand is refused by name when handed one payload. NVFP4 overrides it with the shape-derived split.OperandStore::load,CodecIdentity::admitandExpertEncoding::matrix_bytesnow derive from the codec; the loader's MXFP4 constants and the graph's MXFP4 label each have one home.Behaviour changes
mxfp4is a registered family, so two tests that borrowed it as an "alien" migrated.Stated boundaries
The executor still selects plans itself and emits no requested/selected/reason trace; only CPU realizations are declared; K-quants have no direct realization on the V3 CPU path. Next rung: entropy-coded bf16 as the hostile sixth codec.
Gates
3955 lib tests + all 33 targets and doctests; clippy
--all-targets -D warnings; fmt; doc links; per-file coverage ≥90% on every new file (lowest 93.6%).