Repository navigation
CPU-first OPT2.7B FP16 packets pass bounded acquisition
Pre-releasev0.40.0 - CPU-first OPT2.7B FP16 packets pass bounded acquisition
Fourth completed delivery in the authorized four-to-eight course. The34.122-second
short screen passes faithfulness, resources, capacity, acquisition, runtime and
useful-latency gates. All checked logits match dense streamed FP16 exactly,
including warmups;48 scored generated tokens per condition agree.
Direct dense FP16 stream:80,530,636,800 B /6.487486s. Packets:3,825,246,984 B /
2.295006s, saving95.2499% outgoing H2D and64.6241% episode wall. Both peak at
4,530,999,296 sampled GPU bytes (about4321MiB), under4800MiB from CPU-first
startup. Full FP16 parameter bytes5,303,193,600 alone exceed that allowance.
No hidden full-model GPU preload; current and historical preconstruction allocator
values are zero. Offloading enables BOTH controls; packets make acquisition faster,
not uniquely possible. Optimized Q4/Q8 remains unmeasured.
Protocol,
results and inspirations,
compact receipts,
artifact identity,
source freeze.
Three new32-token prefixes/16-token continuations, consistent FP16 KV, separate
warmups and baseline-first order. Cold worker-entry-to-first DENSE logit13.274175s
is recorded, not represented as packet-first cold latency. Host copies, staging,
metadata, workspace, KV and recording are charged. Artificial budget on a16GiB GPU,
not a small-card OOM test; no FP32-quality, long-context, SSD,32B or deployable-runtime
claim. No fully resident2.7B reference; equality is to the dense streamed reference.
654 CPU tests, exact historical v0.37-v0.39 replays, source reviews and repeat raw
audit pass. Raw scale-v040-raw.zip:30,125,254 B,141 members, SHA256
640b0a5aeee48543e0302518fa2e11219dcebf335e126b4ce649caa644235b95.
Archive restored/hash checked; no model weights redistributed. Final clean-tip
full-suite validation attaches separately.
Complete #55, not
milestone10.
The course establishes FP16/2.7B physical acquisition while preserving failed LRU
and early-forecast economics. Next questions are qualified optimized low-bit
competition and longer-context/block-union durability, each separately frozen.
Four deliveries complete this course; no automatic extra releases, long matrix
or native pivot, and the open ReLU specialist/ranker variants are not declared solved.