Phase-switched loading trades bytes for lower latency

v0.42.0 - Phase-switched loading trades bytes for lower latency

Sixth completed delivery in the authorized four-to-eight course. The 40.671-second
CPU-first OPT-2.7B FP16 screen passes every frozen component gate. All checked
logits match dense streamed FP16 exactly, including both candidate warmups; all
48 scored tokens per condition agree under the same 4800 MiB allowance.

Dense prefill plus exact scalar-decode packets takes 2.545918 s versus 3.223919 s
for always-packet: 21.0303% less whole-episode time and 42.0176% less prefill call
time, while buying 26.1433% MORE outgoing bytes. The cost is explicit. Against
dense streaming, it saves 90.6293% of outgoing H2D and 63.2498% of episode time.
All three modes have the same sampled GPU peak: 4,763,783,168 B (about 4543 MiB).

Protocol,
results and inspirations,
compact receipts,
source freeze.

Known 512-token inputs, 16-token continuations, alternating candidate order and a
baseline-first reference. Whole-episode gain is 0.678001 s: 0.348921 s is in forward
call intervals and 0.329080 s outside them, including recording/episode overhead.
Calls still include in-call recording; this is not trace-free production timing.
Cold shared-worker-to-first DENSE logit is 13.085643 s, not phase-first cold TTFT.
Host copies, staging, workspace, KV, readback, dispatch and records are charged.

No hidden full GPU preload, precision/KV transition, statistical omission or llama.cpp
patch. No new holdout, task-quality, physical-small-card OOM, unique-access, optimized
Q4/Q8, independently resident 2.7B, or deployable-runtime claim. Historical LRU,
forecast and uniform-prefill failures remain failures; this measures a new policy.

All 662 CPU tests and historical raw replays pass; source review and repeat audit
are approved/exact. Raw phase-v042-raw.zip: 79,147,215 B, 160 members, SHA256
97ff97efbbb79923c79f12a60ef09666c7dc4dba4caff1b512c3bdc2efb39f19.
Archive restored/hash checked; final clean-tip full-suite evidence attaches separately.
No model weights redistributed. Prose uses normal spacing; published history and
frozen protocols were not rewritten to clean up older formatting.

Complete #57, not
milestone 10.
Stop this course at six deliveries. Qualified low-bit baselines, instrumentation
overhead and broader context/workload behavior remain open, each requiring a new
short protocol. No automatic long matrix or native pivot.

Longer prefixes separate prefill loss from decode savings

v0.41.0 - Longer prefixes separate prefill loss from decode savings

Fifth delivery, following the owner's continuation request. The 49.901-second
CPU-first OPT-2.7B FP16 screen uses 512-token prefixes and 16-token continuations.
All checked logits are exact, including warmups; 48 scored tokens per condition
match. Both controls fit 4800 MiB, peak sampled GPU 4,763,783,168 B (about 4543 MiB).

Prefill packets use 68.9268% of dense-stream outgoing bytes and 146.3295% of its
forward-call time: BOTH prefill gates fail. Decode uses 3.3287% of bytes and 21.3284%
of call time, passing both decode gates. Whole packet episodes save 92.5714% H2D
and 62.8169% wall (4.187751 s versus 11.262525 s), but this does not waive the prefill
failures. Stop uniform long-prefix expansion; preserve the phase distinction.

Protocol,
results and inspirations,
compact receipts,
source freeze.

Known archived WikiText concatenations, not a new holdout. v0.40 used different
texts, so this is not a paired context-only causal comparison. Same-precision
streamed reference, no full model GPU preload, no FP32-quality/Q4/small-card OOM
claim. Setup, warmups, host duplication, 527-position KV and records are charged.
Cold shared-worker-to-first DENSE logit 17.201395 s is not packet-first cold TTFT.

657 CPU tests, exact historical replays, source review and repeat raw audit pass.
Raw context-v041-raw.zip: 64,918,727 B, 142 members, SHA256
190a02821b42d59a53a706d0045f667caae841124d178095849c8e9ae53df270.
Archive restored/hash checked; final clean-tip full-suite receipt attaches separately.
No model weights redistributed.

Complete #56, not
milestone 10.
The new, unmeasured hypothesis is phase-switched grain: dense prefill transfers,
exact scalar-decode packets, same banks/KV. That requires a NEW short paired
protocol, not optimistic recombination of these timings. No long matrix/native
pivot; optimized low-bit competition remains open.

CPU-first OPT2.7B FP16 packets pass bounded acquisition

v0.40.0 - CPU-first OPT2.7B FP16 packets pass bounded acquisition

Fourth completed delivery in the authorized four-to-eight course. The34.122-second
short screen passes faithfulness, resources, capacity, acquisition, runtime and
useful-latency gates. All checked logits match dense streamed FP16 exactly,
including warmups;48 scored generated tokens per condition agree.

Direct dense FP16 stream:80,530,636,800 B /6.487486s. Packets:3,825,246,984 B /
2.295006s, saving95.2499% outgoing H2D and64.6241% episode wall. Both peak at
4,530,999,296 sampled GPU bytes (about4321MiB), under4800MiB from CPU-first
startup. Full FP16 parameter bytes5,303,193,600 alone exceed that allowance.
No hidden full-model GPU preload; current and historical preconstruction allocator
values are zero. Offloading enables BOTH controls; packets make acquisition faster,
not uniquely possible. Optimized Q4/Q8 remains unmeasured.

Protocol,
results and inspirations,
compact receipts,
artifact identity,
source freeze.

Three new32-token prefixes/16-token continuations, consistent FP16 KV, separate
warmups and baseline-first order. Cold worker-entry-to-first DENSE logit13.274175s
is recorded, not represented as packet-first cold latency. Host copies, staging,
metadata, workspace, KV and recording are charged. Artificial budget on a16GiB GPU,
not a small-card OOM test; no FP32-quality, long-context, SSD,32B or deployable-runtime
claim. No fully resident2.7B reference; equality is to the dense streamed reference.

654 CPU tests, exact historical v0.37-v0.39 replays, source reviews and repeat raw
audit pass. Raw scale-v040-raw.zip:30,125,254 B,141 members, SHA256
640b0a5aeee48543e0302518fa2e11219dcebf335e126b4ce649caa644235b95.
Archive restored/hash checked; no model weights redistributed. Final clean-tip
full-suite validation attaches separately.

Complete #55, not
milestone10.
The course establishes FP16/2.7B physical acquisition while preserving failed LRU
and early-forecast economics. Next questions are qualified optimized low-bit
competition and longer-context/block-union durability, each separately frozen.
Four deliveries complete this course; no automatic extra releases, long matrix
or native pivot, and the open ReLU specialist/ranker variants are not declared solved.

v0.39.0 - Exact FP16 packets save transport and footprint, not resident latency

v0.39.0 - Exact FP16 packets beat streaming, not warm residency

Third of the authorized four-to-eight deliveries. The28.474-second screen passes
FP16 faithfulness/resources/transport/footprint and fails resident-speed. All
checked logits are bit-identical;48 scored tokens per condition match.

Direct FP16 streaming:38,654,705,664 B /3.195469s. Packets:2,161,006,344 B /
1.526528s, saving94.4095% H2D and52.2284% wall. Both offloaded conditions peak at
2,677,116,928 sampled GPU bytes versus resident3,413,217,280 B (21.5662% lower).
This memory saving belongs to offloading, not packet-only residency. Resident
FP16 takes0.382103s and already fits; packets do not beat its warm latency.

Protocol,
results and inspirations,
compact receipts,
source freeze.

Known three32-token prefixes,16-token continuations, declared separate warmups,
FP16 KV and same-precision references. All original-layout host copies, device/
pinned buffers, resource sampling and records are charged. Reference GPU startup
means no CPU-first capacity claim; not FP32-quality preservation or optimized-Q4
competitiveness. Resident-first phase order remains a limitation.

650 CPU tests, unchanged prior raw replays, source review and exact repeat audit
pass. Raw precision-v039-raw.zip:36,303,659 B,155 members, SHA256
b876cb8a0a70ee59b0d636a06c5cbe91b4df0c101f2c088d938b43bac5d564e3.
Archive restored/hash checked; no checkpoint weights redistributed. Final clean-tip
full-suite validation attaches separately.

Complete #54, not
milestone10.
Physical/footprint gates admit a separately frozen short larger sparse-model
capacity experiment; resident-speed failure is preserved. No long matrix/native
pivot follows, and the optimized-Q4 frontier remains open.

v0.38.0 - Early row fetching overlaps compute regions but loses economics

v0.38.0 - Early acquisition overlaps fc1 but fails useful economics

Second of the authorized four-to-eight deliveries. In31.987s including audit,
all checked logits and all48 scored tokens per condition match exactly. Early
same-layer prior-activity forecasts have94.377% event-region overlap with fc1,
but fail traffic and wall gates:37.1284% more H2D and13.2937% slower than packets.

Demand-only:4,266,763,400 B /1.951097s. Identical late forecasts:5,850,946,000 B /
2.158978s. Early:5,850,946,000 B /2.210470s. Forecasts waste193,193 rows and use
188,336. The observed active set forces exact completion before outgoing compute;
wrong predictions waste work, never silently omit contributions. Stop this candidate.

Protocol,
results, inspirations and limits,
compact receipts,
source freeze.

Event brackets include dispatch gaps and do not prove simultaneous arithmetic
kernels. Exposed copy-event wait totals are1.798ms early and1.854ms late; an overlap
counter alone does not establish a useful acquisition policy. Extra forecast GPU
and pinned buffers are8,396,800 B each and charged to every control. Candidate
episodes fit4800MiB, but full-GPU reference startup prohibits a CPU-first capacity
claim. This is OPT1.3B FP32 on three fresh short prefixes, not deployment validation.

646 CPU tests pass; historical v0.37 replay and a second new raw audit match exactly.
Raw early-v038-raw.zip:77,091,994 B,169 members, SHA256
33fb7a8a40986274354dd9464639dffcf122bc7210ac459c4dc328bf7c4b207c.
Every ZIP member restored/hash checked, no weights redistributed. Source and
publication review receipts are retained; final clean-tip tests attach separately.

Complete #53, not
milestone10.
Next freeze stronger precision and scale comparisons. No expanded forecast matrix,
native pivot, Bayesian fitting or stock tuning follows this negative.

v0.37.0 - Exact-row retention: 46% fewer transfers, slower generation

v0.37.0 - Exact row retention saves 46% transfers but loses latency

First of the newly authorized four-to-eight research deliveries. The34.209-second
screen completes with all checked logits bit-identical and all48 scored tokens
per condition matching. Faithfulness, memory and acquisition pass; runtime fails.

1024 retained outgoing rows per layer reduce weight-plus-index H2D from
4,317,841,200 to2,330,056,432 B (46.0365%), but wall rises from1.891826 to4.241058s.
The192MiB cache is charged to both conditions, not a VRAM saving. Forward-call
totals also rise1.593093 to1.944361s: receipt serialization is not the sole loss.
Preserve the negative and stop this LRU candidate before a long matrix.

Frozen protocol,
results and limitations,
course and inspirations,
compact receipts,
source freeze.

Three fresh short prefixes, FP32 OPT1.3B, separate declared warmups, imposed
4800MiB candidate budget and6500MiB reference allowance. Full reference GPU startup
means NO CPU-first capacity claim. Not stronger precision, larger-model deployment,
task-quality qualification or native-pivot evidence. v0.31/v0.34 used Qwen, not
ReLU; their negatives do not close ReLU predictive approaches.

639 CPU tests pass and a second raw audit matches exactly. Pre-inference review
caught an exact-boundary comparison bug; integer byte comparisons and a regression
test fixed it before measurement, without relaxing gates. Raw archive
retention-v037-raw.zip:70,387,510 B,151 members, SHA256
4c7b55de52096c9a27bfbc39bd7cde3f5180c96c2be3e924f078316ee95d9bdd.
Every member is restored/hash checked; checkpoint weights are not redistributed.
Final clean-tip full-suite validation is attached separately.

Complete #52, not
milestone10.
Next: separately freeze causal early acquisition with exact demand completion,
then stronger precision and scale comparisons. No stock tuning or native patch.

v0.36.0: FP16 access limits and six-release research findings

v0.36.0 - FP16 fits and matches tokens; first-use latency misses the gate

Sixth/final authorized delivery. The31.048-second screen is complete and audited.
Ordinary resident FP16 fits below4800MiB and matches ALL16 own-trajectory greedy
tokens. Its compound useful-access gate fails because the first episode takes
13.067s, above5s; the second takes0.062s. Preserve the first-use stall: no warmup
discard, repeat, relaxed threshold or attributed cause without profiling.

FP32 numerical fidelity separately fails: aligned full-logit relativeL2 ranges
0.000615--0.005117 against1e-5. IDs still match. This is numerical difference, not
demonstrated task-quality loss. Peak sampledGPU3,406,925,824 B; parameters
2,631,516,160 B and maximum FP16 KV4,521,984 B. All physical/state/resource audits
pass. Setup2.302s and constructionH2D2,631,516,160 B are charged separately; there
is no weight acquisition during generation.

Protocol,
results and inspirations,
six-release findings,
compact receipts,
source freeze.

The course's strongest result is the exact outgoing-weight packet loader on ReLU
OPT: v0.35 stays below4800MiB from CPU-only startup and saves93.65%H2D/72.16%wall
versus stronger contiguous FP32 streaming. FP16's observed capacity/token parity
and first-use stall do not establish uniquely enabled access or a paired warmed
speedup for packets. Known short prefixes, an imposed budget and small sample do
not qualify a larger checkpoint, task quality, long context or deployment.

622 CPU tests pass; exact second audit adds no inference. Final clean-tip full-suite
receipts are attached separately. Raw resident-v036-raw.zip:6,410,382 B,
115 members, SHA256 acbe26d172b01687a2cdaa7901e7d573a7b078696ddbd49cf68c1b4451e1730b.
Archive restored/hash checked; external references included, checkpoint weights not
redistributed. Earlier negative results and the first packet failure stay preserved.

Complete #51 and
the six requested releases, NOT milestone10.
No additional sweep, long matrix, model download or native pivot starts automatically.

v0.35.0: CPU-first packets pass bounded FP32 capacity

v0.35.0 - CPU-first packets pass a bounded FP32 capacity screen

Fifth of six deliveries. All four gates pass in23.033s including audit. The model
starts entirely on CPU and never becomes fully GPU-resident. Under a declared
4,800MiB GPU ceiling, all checked logits and32 generated tokens match the dense
FP32 reference exactly.

Condition	Tokens	Weight + index H2D	Wall
Direct contiguous dense streaming	16	25,769,803,776 B	2.646471s
Exact-active-row packets	16	1,635,580,200 B	0.736909s

93.65% fewer bytes and72.16% lower aggregate wall survive a stronger dense control
that avoids per-call layout conversion. Full parameters require5,263,032,320 B,
above the5,033,164,800 B allowance. Peak sampled total GPU4,380,004,352 B and peak
allocated/reserved3,805,911,552 /3,827,302,400 B remain below it.

Protocol,
findings, costs and inspirations,
compact receipts,
source freeze.

Both methods fit; the result is useful acquisition under the same FP32 contract,
not novelty in offloading alone. Two known16-token prefixes, eight-token caps and
one counterbalanced pair do not establish task quality or a general speedup.
References are immutable CPU-only arrays from v0.33. This imposed budget on a16GiB
GPU is not a physical small-GPU/OOM test, larger checkpoint, cold SSD result or
proof against every sub-sample driver transient. Ordinary resident FP16 remains
an untested stronger deployment comparison for the final delivery.

All host layouts,134,283,264 B pinned buffers, packet/workspace, KV and setup are
charged.604 CPU tests and exact repeat audit pass; final clean-tip validation is
attached separately. Raw capacity-v035-raw.zip:12,105,512 B,124 members, SHA256
c546d19fe7156786b3a6ec9f5351482b4b4f11ac2bd53e529dc4ab3d1d2bafdb.
Archive restored/hash checked; no checkpoint weights redistributed.

Complete #50, not
milestone10.
One release remains. No long matrix, native pivot or production claim follows.

v0.34.0: Full-vocabulary acquisition loses actual-prefix economics

v0.34.0 - Full-vocabulary risk does not buy cheaper accepted prefixes

Fourth of six authorized deliveries. The84.876-second screen preserves all target
outputs and passes faithfulness, but fails acquisition and runtime. Stop this
full-vocabulary ranker; do not expand the matrix.

Physical acquisition	Accepted / proposed	H2D / accepted token	Wall
Binary-pair risk17	15/16	627,100,911 B	9.558s
Full-vocabulary risk17	13/20	837,240,706 B	35.847s
All35 pages	15/16	644,087,842 B	9.532s

Joint Gaussian page/remaining-field samples score all151936 vocabulary competitors,
then real selected pages are purchased and the independent dense target verifies
actual K4 prefixes. More rejection causes29 draft calls versus24 per control; the
controller alone costs25.928s. All48 committed tokens match. This is not a calibrated
acceptance posterior, cross-token budget optimizer or omission certificate.

Protocol,
results and inspirations,
compact receipts,
source freeze.

Two new four-token prefixes and eight-token caps are a short mechanics/economics
screen, not task quality or a capacity frontier. The prior is reused unchanged.
All28 numerical controls and explicit separate-KV/crop rules pass. Extra CUDA
allocation1,009,306,112 B remains below1GiB, with totalGPU peak9,394,294,784 B.
589 CPU tests pass; exact second audit adds no inference. Final clean-tip full-suite
validation is attached separately. Original v0.30 verdict/replay stays unchanged.

Raw vocabulary-v034-raw.zip:56,826,528 B,130 members, SHA256
b03b96f31538fa1c29adf1ff259f6bad96f42461813452733176bf91f913c5ac.
Archive restored/hash checked; checkpoint weights are external, not redistributed.
The sampled-ID audit does not independently repeat every neural head evaluation.

Complete #49, not
milestone10.
Two releases remain, focusing on packet capacity and a stronger low-memory control.
No further ranking of these pages, long matrix, native pivot or production claim.

v0.33.0 - Compact exact-active-row packets pass the short screen

v0.33.0 - Compact exact-active-row packets pass the physical-loading screen

Third of six authorized deliveries. The corrected 25.177-second screen passes
both gates. Actual weight-plus-index H2D falls 93.65%, and wall time falls
69.36% against dense streaming. Both comparisons also beat128-row sparse
acquisition. Every checked full-logit value and emitted token matches the original
dense reference exactly on the completed corrected workload.

Condition	Tokens	Weight + index H2D	Wall
Streaming	16	25,769,803,776 B	2.142871s
128-row sparse extents	16	22,376,611,840 B	2.088562s
Compact active-row packet	16	1,635,580,200 B	0.656515s
Restored resident	16	0 B	0.168545s

Compute the first projection fully, observe exact zeros, gather active outgoing
rows plus indices into one packet, transfer once and scatter into the original
dense workspace. Dense linear computation remains; this is real acquisition, not
a proxy traffic simulation, learned selector, custom kernel or llama.cpp flag study.
Packet is3.26x as fast as this streaming control but still3.90x slower than the
restored resident. Two short pretrained-OPT continuations do not establish task
quality, general speedup, long-context behavior or larger-model access.

The first attempt stopped after18.089s: IDs matched, but one logit error exceeded
the unchanged1e-5 tolerance. Its six partial episodes remain inconclusive and are
archived. A reviewed fresh protocol restored original weight layout and F.linear
for ALL conditions; prompts/order/gates stayed identical. Corrected run19.111s +
audit6.066s passes, with zero logit error in six hybrid and two restored episodes.
The prompts are known/reused after that failure, not an untouched holdout. Exact
second audit uses no new inference. 579 CPU tests pass; final clean-tip XML and
validation are attached separately.

First protocol,
prospective correction,
full findings, costs and inspirations,
compact receipts,
corrected source freeze.

Packet indices, CPU gathering, GPU scattering, extra buffers, original weights,
setup/restoration, KV, readbacks and raw writes are charged. Total pinned buffers
134,283,264 B; packet GPU67,174,400 B plus workspace67,108,864 B. Peak sampled GPU
5,990,617,088 B and RSS9,047,523,328 B. Original full residency and retained allocator
reservation still prevent a hard-budget capacity claim. Corrected strided controls
pay framework rearrangement; this is not the best possible native/pretransposed
streaming baseline, nor is FP32 the strongest low-memory deployment baseline.

Main raw archive packet-v033-raw.zip:32,528,344 B,138 members, SHA256
c295f0953136c3489da23a9e08a0f8d83e5e18a3d2d74e3efb6f9f2c26e41bbd.
Includes the failed raw archive and receipt:12,356,618 B,114 members, SHA256
968b370f2fb5851f95af4767a787dd0ff5287af32efb54a6d67828a700c48933.
Both archives restored/hash checked; no checkpoint weights redistributed.

Complete #48, not
milestone10.
Nominate the packet mechanism for a separately frozen follow-up. Three deliveries
remain, including accepted-prefix risk and useful capacity. No long matrix,
native qualification, production claim or rewriting of prior negatives follows.