Skip to content

v0.33.0 - Compact exact-active-row packets pass the short screen

Pre-release
Pre-release

Choose a tag to compare

@displague displague released this 15 Sep 14:01
· 18 commits to main since this release

v0.33.0 - Compact exact-active-row packets pass the physical-loading screen

Third of six authorized deliveries. The corrected 25.177-second screen passes
both gates. Actual weight-plus-index H2D falls 93.65%, and wall time falls
69.36% against dense streaming. Both comparisons also beat128-row sparse
acquisition. Every checked full-logit value and emitted token matches the original
dense reference exactly on the completed corrected workload.

Condition Tokens Weight + index H2D Wall
Streaming 16 25,769,803,776 B 2.142871s
128-row sparse extents 16 22,376,611,840 B 2.088562s
Compact active-row packet 16 1,635,580,200 B 0.656515s
Restored resident 16 0 B 0.168545s

Compute the first projection fully, observe exact zeros, gather active outgoing
rows plus indices into one packet, transfer once and scatter into the original
dense workspace. Dense linear computation remains; this is real acquisition, not
a proxy traffic simulation, learned selector, custom kernel or llama.cpp flag study.
Packet is3.26x as fast as this streaming control but still3.90x slower than the
restored resident. Two short pretrained-OPT continuations do not establish task
quality, general speedup, long-context behavior or larger-model access.

The first attempt stopped after18.089s: IDs matched, but one logit error exceeded
the unchanged1e-5 tolerance. Its six partial episodes remain inconclusive and are
archived. A reviewed fresh protocol restored original weight layout and F.linear
for ALL conditions; prompts/order/gates stayed identical. Corrected run19.111s +
audit6.066s passes, with zero logit error in six hybrid and two restored episodes.
The prompts are known/reused after that failure, not an untouched holdout. Exact
second audit uses no new inference. 579 CPU tests pass; final clean-tip XML and
validation are attached separately.

First protocol,
prospective correction,
full findings, costs and inspirations,
compact receipts,
corrected source freeze.

Packet indices, CPU gathering, GPU scattering, extra buffers, original weights,
setup/restoration, KV, readbacks and raw writes are charged. Total pinned buffers
134,283,264 B; packet GPU67,174,400 B plus workspace67,108,864 B. Peak sampled GPU
5,990,617,088 B and RSS9,047,523,328 B. Original full residency and retained allocator
reservation still prevent a hard-budget capacity claim. Corrected strided controls
pay framework rearrangement; this is not the best possible native/pretransposed
streaming baseline, nor is FP32 the strongest low-memory deployment baseline.

Main raw archive packet-v033-raw.zip:32,528,344 B,138 members, SHA256
c295f0953136c3489da23a9e08a0f8d83e5e18a3d2d74e3efb6f9f2c26e41bbd.
Includes the failed raw archive and receipt:12,356,618 B,114 members, SHA256
968b370f2fb5851f95af4767a787dd0ff5287af32efb54a6d67828a700c48933.
Both archives restored/hash checked; no checkpoint weights redistributed.

Complete #48, not
milestone10.
Nominate the packet mechanism for a separately frozen follow-up. Three deliveries
remain, including accepted-prefix risk and useful capacity. No long matrix,
native qualification, production claim or rewriting of prior negatives follows.