BALANCED-SEARCH-2: evidence-aware representation search semantics - #366
Merged
Conversation
…a scalar
BALANCED-SEARCH-2 rung 1. Ranking representation candidates by "bytes
saved per unit KL" treats the remaining behavioural budget as one pool.
The four-family Kimi map measured otherwise at 8192 positions under
kimi-logit-balanced-v1:
kl p99 68 % of limit plenty
top-1 mass displaced 46 % of limit plenty
top-10 mass p99 54 % of limit plenty
route mixture p99 83 % of limit BINDING
route mixture max 80 % of limit BINDING
A candidate that looks cheap on logits but moves routing spends the
resource that is actually scarce. `ConstraintVector::binding()` names
that resource, so a search can rank against it instead of against KL.
The type also separates the two kinds of limit a gate carries, because
this programme has now been bitten by conflating them. kl, the
displacement limits and the raw counts are CEILINGS a candidate spends
against. positions and covered mass are FLOORS the MEASUREMENT must
clear. A blind instrument scores perfectly on every ceiling — that is
what `kl_p99 = 0.0` means — and is caught only by a floor. So
`binding()` ranks ceilings only and `sound()` asks about floors
separately, with a test asserting that a constant-logit bank satisfies
every budget and is still refused.
`observed: None` is not zero. An unmeasured consequence leaves a
criterion unranked and unsatisfied, except where the bank recorded no
change of that kind at all and there was no magnitude to record —
`Margin::vacuous`, mirroring the gate's own rule.
The margin table is a second reading of the same limits `evaluate`
applies, so a congruence test pins the two together over five banks
including both real 8192-position measurements and a flat-instrument
one: the vector and the gate must agree on the verdict and criterion
for criterion. Drift fails a test rather than silently mis-ranking.
Each consequence magnitude sits in the table beside the COUNT whose
being zero makes its absence vacuous, rather than in a second function
that re-derives the pairing from the criterion and needs a wildcard arm
to do it. The wildcard was also structurally dead — the helper was only
ever reached for the three displacement criteria — and a reader can now
check the correspondence in one place.
A zero-valued ceiling is judged but never ranked. No gate in the crate
sets one today, and the type does not assume that stays true:
`observed / 0.0` is infinity, or NaN when nothing was observed either,
and a NaN utilisation sorts arbitrarily in `max_by` — it would
silently mis-rank the whole vector rather than fail.
Fixtures are the measured banks, not invented ones —
kimi_full4-{selection,heldout}-8192 and one of the two catalogued
blind-instrument episodes. The overspend case uses B3, the map
balanced-v1 was calibrated to refuse.
Gates: fmt --all, check --all-targets, check --examples, clippy
--all-targets -D warnings, e0 boundary, tests (lib + all integration
binaries), benches. Coverage: constraint.rs 100.00% (148/148) against
the 90% per-file floor; crate total 93.37%.
… beta is measured, not assumed
BALANCED-SEARCH-2 rung 2, the physical half of the objective. Three
things that are routinely conflated are kept apart, because only the
first is intrinsic:
physical fact bytes/token for a representation map
observation this byte reduction produced this GPU-time
reduction, on this machine, at this breadth
planner coefficient beta = gpu fraction removed / byte fraction
removed
byte_ledger.rs holds the fact. A ledger lists EVERY scope the decoder
reads per token, changed or not — one that listed only the changed
scopes could not say what fraction of the whole moved, which is the
single quantity a throughput prediction is a function of. Scopes are
supplied rather than inferred from geometry: "which layers are MLA" is
a model-shape question whose wrong answer would be silent and would
propagate into every prediction downstream.
execution_cost.rs holds the observation, with provenance — machine,
device, backend, the commit the measured binary was built from (a
kernel change moves beta), model identity, both representations, the
protocol, and the evidence files. Breadth is explicit: families
changed and scopes changed, so that beta(breadth) becomes a question
that can be ASKED of an accumulated set rather than assumed flat now.
The derived quantities are methods, not fields, so a stored record
cannot disagree with its own inputs.
Writing `const BETA: f64 = 0.80` would promote one measurement to a law
of the backend. This programme has already paid for that once: a ~45 GB
wired-memory wall inferred from a single episode was contradicted at
79 GB the next day. So the model reports Provisional and says why.
Calibration status is about BREADTH, not count: ten observations at the
same byte fraction agree with each other and say nothing about whether
beta varies, so Calibrated requires separated operating points. With
one measurement the status is Provisional and every prediction carries
it — `predicted 40.9 tok/s using provisional execution-cost observation
m3max-metal-001`. A prediction at a byte fraction far from any measured
one is flagged Extrapolated and names the nearest breadth that was
actually measured.
No regression, no confidence interval, no piecewise curve: with one
point those are decoration. The arithmetic is
`gpu_removed = beta x bytes_removed` and nothing more. When there are
five or ten observations the model can learn a curve without the search
API changing.
There is no way for search code to receive a naked coefficient. A
CostPrediction is obtainable only from ExecutionCostModel::predict,
which takes a ledger, checks the model identity matches (byte economics
do not transfer across models) and refuses with a typed CostRefusal
naming the measurement to go and take.
One assumption is pinned rather than hidden: fixed_overhead_ms is a
single number for both arms, so it assumes overhead does not depend on
the map. The bench measured 1.05 ms and 1.09 ms on the two arms, worth
about 0.1% on a predicted wall speedup — 1.140 against a measured
1.139. A test asserts both, so a backend that makes overhead
map-dependent will say so.
m3max_metal_001 records the measurement behind all of it: 957.0 MB of
5.985 GB removed (15.99%), GPU 26.87 -> 23.43 ms (12.80%), beta 0.8006,
with session 2 independently at 0.7907.
Gates: fmt --all, check --all-targets, check --examples, clippy
--all-targets -D warnings, e0 boundary, tests, benches, per-file
coverage.
…ly, against what remains
BALANCED-SEARCH-2 rung 3, the bridge between the two halves. constraint
says which behavioural resource is scarce; execution_cost says what a
map buys in decode time; this says what one MOVE buys and what fraction
of the scarce resource it consumes.
The objective is unchanged and is not this: maximise predicted
throughput subject to the contract passing. What lives here is the
heuristic that picks which move to try next, which is a different thing
and is documented as such.
**Everything is marginal.** A candidate leaving routing at 88% has not
"cost 88%": the map was already at 83%, so the move took five of the
seventeen points that remained — 29% of the budget actually available.
Absolute utilisation cannot express that, and a search ranking on it
would keep choosing moves that eat the scarce resource while reporting
a comfortable-looking number.
The ranking is predicted GPU ms saved over the largest fraction of any
criterion's REMAINING headroom the move consumed. The test that
justifies the normalisation: a move that FREES the binding constraint
and costs one point of KL (of 32 remaining) outranks one that costs no
KL at all and takes three points of routing (of 17 remaining). A naive
"cost = absolute KL delta" ranks these the wrong way round, and the
test asserts that it would.
**The vector survives the score.** ranking_score is derived
convenience; before, after and the marginal vector are the evidence, so
"it freed routing" remains readable rather than collapsed. The binding
criterion is rechecked after the move and can change — there is a test
where it does, because a search must not assume what binds is constant.
**A ranking is not an admission.** EvidenceScale::Diagnostic can be
scored and sorted and produces Admission::Estimated; only Authority can
produce Earned. The position floor is scale-dependent, since a
diagnostic bank is short by definition and failing it says nothing
about the candidate. Coverage is NOT scale-dependent: a blind
instrument scores perfectly on every ceiling, and a ranking computed
from one is worthless in exactly the way an admission from one would
be. Tested at both scales.
**`None` must never acquire an accidental economic meaning.** Four
states arrive that a scalar would collapse, and MoveClass keeps them
apart because two of them look identical from the outside:
Unpriced buys time, consumes no measurable headroom
Priced buys time at a measurable cost
Unscorable buys time, but a cost could not be scored — a
criterion was already over budget, or evidence is
missing on one side
Worthless buys no time, however cheap
Unpriced and Unscorable BOTH surface as
`scarce_fraction_consumed == None` and rank at opposite ends. Reading
one as the other put a move that had blown past an exhausted budget at
the TOP of the ranking; a test pins them in different tiers.
The order is total and written down rather than emergent: tiers first,
then gain per unit of scarce headroom within Priced and physical gain
elsewhere, then frugality, then candidate identity. `total_cmp` leaves
no pair incomparable, so a sort cannot depend on input order — a search
trace that cannot be reproduced from its inputs is not evidence.
describe() renders the trace a reader can argue with:
candidate: removes 16.0% -> removes 20.0%
physical:
-239 MB/token
predicted -0.86 ms GPU
behavioural:
kl p99 +6% of remaining headroom
routed mixture moved at p99 +29% of remaining headroom <- scarce resource
rank: 2.93 ms per unit of scarce headroom (29% consumed)
evidence: Diagnostic, Estimated
Gates: fmt --all, check --all-targets, check --examples, clippy
--all-targets -D warnings, e0 boundary, tests, benches, per-file
coverage.
…scale that supplies its tail
A p99 over forty-six observations is a maximum wearing a percentile's
name. Nearest-rank p99 of n values IS the largest whenever n < 100, and
a search that prices a candidate on such a number is pricing a single
event.
tail support = observations x (1 - quantile)
p99 over 100 observations -> 1 expected tail observation
p99 over 500 observations -> 5
p99 over 1,000 observations -> 10
Found the expensive way. ROUTE-CAL-1 (2026-08-31): a 256-position
diagnostic bank observes 1-105 route-change events, so its
route_mass_p99 was the maximum in twenty-four of twenty-six reports,
and repeated values across unrelated maps were the SAME worst event
recurring rather than candidates agreeing. The consequence was not
noise but a sign error — the diagnostic reported the most expensive
route move in the programme as free.
MeasurementStatus keeps three states apart that a scalar would collapse:
Measured, InsufficientTailSupport{observations, required} and
NotObserved. A thinly supported percentile is not a small cost, it is
an unknown one, which is what stops a search preferring candidates
whose expensive dimension happens to be unmeasured.
The threshold is a POLICY carrying its own provenance, not a constant:
five expected tail observations, chosen because one observation
defining a tail is technically a percentile and useless for ranking.
A refusal can then be traced to the decision that refused it, and
tightening it later is a visible change.
The abstraction is not about routing. Any p95, p99 or p999 criterion
screened at reduced scale inherits the protection without anyone
remembering to ask.
Gates: fmt --all, check --all-targets, clippy --all-targets -D
warnings, e0 boundary, tests, per-file coverage.
…d predictive utility are orthogonal
measurement.rs answers whether a statistic was measured, and answers it
strictly. This answers a different question: what a search may DO with
one that was not.
the STATISTIC decides whether it was measured
the EVIDENCE decides whether an imperfect statistic is still useful
Collapsing them is a specific trap. Diagnostic kl p99 is thin-tailed
(2.56 expected tail observations at 256 positions) AND correlates with
authority ordering at rho +0.857 over seven paired maps. It would be
tempting to call that "Measured, provisionally". It is not: rho +0.857
is evidence for "A or B first?", not evidence that a diagnostic reading
of 2.4e-3 means authority will be near 2.4e-3 — which is exactly what
pricing it against a 3.5e-3 budget assumes. The same programme measured
promotion drift of +84% on a narrow map and -3.9% on a broad one.
Four rungs, of which only the top two may produce a number a contract
is priced against:
Direct enough support to price against the contract
CalibratedEstimate magnitude transfer demonstrated across breadths
OrderingProxy useful for ordering; magnitude not trusted
Unusable no demonstrated search value
A registration wins in BOTH directions. It can raise a thin measurement
to an ordering proxy, and it can LOWER a perfectly well measured one:
route flip rate has no tail problem at all and is still only a proxy,
because the contract judges mixture mass and not flips. Without a
registration the measurement decides, and an unregistered thin
percentile is Unusable rather than usable — the failure this layer
exists to prevent is a search preferring a candidate because its
expensive dimension happened to be unmeasured.
The registry starts from what ROUTE-CAL-1 established, and every entry
names its evidence: diagnostic kl an OrderingProxy at rho +0.857;
diagnostic route mass p99 Unusable, having called the programme's most
expensive route move free and got the sign wrong on two of five;
diagnostic route flip rate an OrderingProxy at rho +0.991, a COUNT
statistic surviving the sample size where a TAIL statistic does not;
diagnostic top-10 mass p99 Unusable by ABSENCE of calibration, worded
so it reads as a measurement someone should run rather than a property
of the statistic.
The payoff is that `Unscorable` stops meaning "the search knows
nothing". It means this authority constraint cannot be priced
numerically at this evidence scale, while the search may still hold
calibrated proxy evidence about which candidate to measure next.
Nothing is wired into CandidateAssessment yet, and balanced-v1 is
unchanged.
Gates: fmt --all, check --all-targets, clippy --all-targets -D
warnings, e0 boundary, tests, per-file coverage.
…dimension may not win by being invisible
The last semantic step before a search loop. Three questions that were
at risk of collapsing into one are now answered by three types:
was the statistic measured? MeasurementStatus
can it be used predictively? SearchEvidence
can this candidate be priced? CandidateAssessment
`Margin` carries the tail support behind each percentile — the
observation count, or `None` for a statistic that is not a percentile
and has no tail to be thin. `MarginalConstraintCost` carries the
`SearchEvidence` for its dimension, taken as the WEAKER of the two
sides: a move whose "after" is well supported but whose "before" was
not has no trustworthy delta. Only priceable dimensions may set the
scarce fraction, so at diagnostic scale under ROUTE-CAL-1 nothing is
priced and every candidate is `Unscorable` — correctly, since kl is an
ordering proxy there and route mass p99 is unusable.
That exposed the statistical twin of the exhausted-budget bug. Two
candidates whose route cost is equally unpriceable tier equally and
then sort on physical gain, so the one that moves routing hardest wins
BECAUSE nothing could see it. promotion.rs closes it with a class, not
a score:
Priceable every judged dimension was priceable
ProxySupported unpriceable dimensions have benign ordering evidence
ProxyRisky a proxy warns
Uninformed an unpriceable dimension nothing can speak to
(worthless ranks last whatever its evidence)
A dimension whose OWN evidence orders is its own proxy — diagnostic kl
cannot be priced and can still say which of two candidates is worse.
Only a dimension nothing can order needs an external stand-in, which is
route mass p99 and its flip-rate proxy.
The proxy changes the CLASS and never the score. Turning ROUTE-CAL-1's
rho +0.991 flip-rate result into an approximate route-mass number would
be exactly the conflation of ordering with magnitude that the ladder
exists to prevent; a test asserts no route price appears anywhere in an
assessment however large the proxy delta.
The invariant, executable: A buys more, costs less on the only priced
dimension, and would win on economics alone — and must not, because its
flip-rate proxy is elevated while B's is benign. Also tested: a warning
outranks silence (Uninformed is worse than ProxyRisky, because a
warning is information); an `Unusable` observation offered AS a proxy
is not laundered into evidence; the weakest unpriceable dimension
decides rather than the average; the order is total and reproducible.
`EvidenceScale` moves to measurement.rs, where it belongs — a scale is
what a statistic's support is judged at, and the calibration registry
must key on it without depending on anything that consumes it.
balanced-v1 is unchanged and `CalibratedEstimate` remains unpopulated:
no magnitude-transfer evidence exists yet, and the variant is worth
more empty than guessed at.
Gates: fmt --all, check --all-targets, clippy --all-targets -D
warnings, e0 boundary, tests (3559 lib), benches.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
The semantics of an evidence-aware representation search. Six commits, each
independently green, together forming one foundation — and deliberately no
consumer: the first real-bank integration is the next rung, not this PR.
One invariant runs through all of it:
The six
b3b1deafc88265d072776ba4486eef4bc145136cbe460e40Why each exists
The budget is a vector. Measured on the four-family Kimi map at 8,192
positions: kl at 68 % of limit, top-1 mass 46 %, top-10 54 % — and route mixture
mass at 83 %. Ranking on "bytes per unit KL" spends the resource that is not
scarce.
binding()names the one that is.Ceilings and floors are different things. kl, displacement and counts are
budgets a candidate spends; positions and covered mass are conditions on the
MEASUREMENT. A blind instrument scores perfectly on every ceiling —
kl_p99 = 0.0is a perfect KL — and is caught only by a floor. That happened twice whilethe four-family map was being earned.
Beta is an observation, not a constant. 957 MB of 5.985 GB removed (16.0 %)
bought 12.8 % of GPU time — beta 0.80. Writing
const BETA = 0.80would promoteone measurement to a law of the backend; this programme already paid for that
once with a ~45 GB wired-memory wall that was contradicted at 79 GB the next day.
ExecutionCostModelreports Provisional and says why, and calibration statusis about BREADTH rather than count: ten observations at the same byte fraction
establish repeatability, not transferability.
Everything marginal. A candidate leaving routing at 88 % has not cost 88 % —
the map was at 83 %, so it took five of the seventeen points that remained. The
test that justifies it: a move that FREES the binding constraint at a cost of one
KL point (of 32 remaining) outranks one costing no KL at all that takes three
route points (of 17). A naive absolute-KL ranking inverts that, and the test
asserts it would.
A percentile needs a tail. Nearest-rank p99 of n values is the LARGEST
whenever n < 100. ROUTE-CAL-1 found a 256-position bank observing 1–105
route-change events, so its
route_mass_p99was the maximum in 24 of 26 reports— and it reported the most expensive route move in the programme as free.
Measurement adequacy and predictive utility are orthogonal. Diagnostic kl is
thin-tailed AND correlates with authority ordering at ρ +0.857. That is evidence
for "A or B first?", not evidence that a diagnostic 2.4e-3 means authority will
be near 2.4e-3 — which is what pricing against a 3.5e-3 budget assumes. Four
rungs, of which only the top two may produce a number a contract is priced on:
Direct,CalibratedEstimate,OrderingProxy,Unusable. A registration winsin both directions: route flip rate has no tail problem at all and is still only
a proxy, because the contract judges mixture mass and not flips.
An unpriced dimension may not win by being invisible. Two candidates whose
route cost is equally unpriceable tier equally and then sort on physical gain —
so the one that moves routing hardest wins because nothing could see it.
promotion.rscloses it with a class, never a score.Deliberately not done
balanced-v1is unchanged. No goalposts moved after seeing search data.CalibratedEstimateis unpopulated — no magnitude-transfer evidence exists,and the variant is worth more empty than guessed at.
top-10 mass p99is registeredUnusablewithpairs: 0andwording that reads as a measurement someone should run, not a property of the
statistic. A test asserts that wording.
the next rung, and it is the first thing neither reasoning nor fixture coverage
can settle: a fixture written by the author of
ConstraintVector::ofcan encodethe same mistaken assumption twice and stay green.
Gates
fmt --all·check --all-targets·check --examples·clippy --all-targets -D warnings·e0_generation_boundary·cargo test -p larql-vindex(3559 lib + all integration binaries) ·benches.Per-file coverage: constraint 100 %, execution_cost 99.3 %, assessment 98.9 %,
byte_ledger 97.9 %; crate 93.4 % against a 71 % floor.
Each of the six was verified green independently before commit.