Skip to content

v0.35.0: CPU-first packets pass bounded FP32 capacity

Pre-release
Pre-release

Choose a tag to compare

@displague displague released this 15 Sep 14:50
· 14 commits to main since this release

v0.35.0 - CPU-first packets pass a bounded FP32 capacity screen

Fifth of six deliveries. All four gates pass in23.033s including audit. The model
starts entirely on CPU and never becomes fully GPU-resident. Under a declared
4,800MiB GPU ceiling, all checked logits and32 generated tokens match the dense
FP32 reference exactly.

Condition Tokens Weight + index H2D Wall
Direct contiguous dense streaming 16 25,769,803,776 B 2.646471s
Exact-active-row packets 16 1,635,580,200 B 0.736909s

93.65% fewer bytes and72.16% lower aggregate wall survive a stronger dense control
that avoids per-call layout conversion. Full parameters require5,263,032,320 B,
above the5,033,164,800 B allowance. Peak sampled total GPU4,380,004,352 B and peak
allocated/reserved3,805,911,552 /3,827,302,400 B remain below it.

Protocol,
findings, costs and inspirations,
compact receipts,
source freeze.

Both methods fit; the result is useful acquisition under the same FP32 contract,
not novelty in offloading alone. Two known16-token prefixes, eight-token caps and
one counterbalanced pair do not establish task quality or a general speedup.
References are immutable CPU-only arrays from v0.33. This imposed budget on a16GiB
GPU is not a physical small-GPU/OOM test, larger checkpoint, cold SSD result or
proof against every sub-sample driver transient. Ordinary resident FP16 remains
an untested stronger deployment comparison for the final delivery.

All host layouts,134,283,264 B pinned buffers, packet/workspace, KV and setup are
charged.604 CPU tests and exact repeat audit pass; final clean-tip validation is
attached separately. Raw capacity-v035-raw.zip:12,105,512 B,124 members, SHA256
c546d19fe7156786b3a6ec9f5351482b4b4f11ac2bd53e529dc4ab3d1d2bafdb.
Archive restored/hash checked; no checkpoint weights redistributed.

Complete #50, not
milestone10.
One release remains. No long matrix, native pivot or production claim follows.