Repository navigation
v0.4.17
The repack cache is 1.9 per cent larger than the model file where it was ten per cent, so an 8B holds 4781 MiB of weights against 5151 and a 4090 holds 5540 MiB of them at a context of 2048 against 5910. That is 1.066 times what llama.cpp holds on the card, and it costs 2.5 per cent of a decode. There is no longer a lossy step anywhere in the repack. All eight quantized types round trip bit for bit and the logit corpus agrees with llama.cpp on all thirteen device cases on both backends.