Skip to content

v0.5.5

Latest

Choose a tag to compare

@SimJeg SimJeg released this 17 Sep 12:45
· 10 commits to main since this release
a13a1da

New Features

  • Add RestoreKVPress — learned, budget-matched restoration on top of KVzip using restore tokens and LoRA adapters, #259 by @higokri
  • Add EntropyGatedChunkKVPress — entropy-aware chunk compression that rescues important tokens from spiky chunks, #263 by @ShaharBenIshay
  • Add KVgradPress — gradient-based KV importance scoring built on KVzip replay, #279 by @joshua840

Bug Fixes

  • Fix cache-position phase detection by adding a shared is_prefilling helper, #243 by @fallintoplace
  • Harden checkpoint loading and unsafe deserialization paths, #260 by @SimJeg
  • Fix PyramidKVPress budget underflow and division by zero, #266 by @AmirF194
  • Fix silently ignored num_logits_to_keep in the pipeline and KVzipPress, #269 by @emecii
  • Fix KVzap checkpoint loading with Transformers 5, #270 by @SimJeg
  • Fix nested boxed-answer parsing, #271 by @alessiodevoto
  • Harden evaluation against remote code execution, #272 by @SimJeg
  • Prevent scorer presses from producing empty caches on short contexts and complete minimum-budget handling, #264 and #277 by @SuperMarioYL
  • Fix random scoring device and dtype mismatches, #273 by @emecii
  • Compute KnormPress scores in float32 to avoid low-precision ties, #280 by @jg-codes
  • Fix fp16 attention patch overflow and DuoAttention QK normalization, #283 by @SimJeg

Documentation