Skip to content

v2.0 - Workspace-scored eviction (J-lens)

Latest

Choose a tag to compare

@Anyesh Anyesh released this 09 Jul 14:59

EVOKE gains a forward-looking, content-based eviction signal distilled from the Jacobian-lens workspace readout (Gurnee et al., 2026): a per-layer ridge probe scores KV blocks at prefill time, one dot product per position, before any decode history exists.

Highlights:

  • JLensScorer as a fifth first-class signal (EVOKE_W_JLENS, EVOKE_JLENS_PROBE), composing with kv_restore recovery.
  • Validated on pre-registered gates across three Qwen generations: fact-AUC 0.891/0.865/0.939 vs SnapKV 0.622/0.563/0.541 (Qwen2.5-7B / Qwen3-8B / Qwen3-4B); masked eviction at 25% retention keeps the probed fact in 35-36 of 36 episodes vs SnapKV's 25/14/13.
  • agent_bench: jlens passes every budget with recovery disabled while H2O fails all three; measured decode overhead -0.6% (noise).
  • Fork residual-capture API exported with C linkage (llama_set/get_embeddings_layer_inp), per-arch layer-input registration for qwen2, fork synced with upstream llama.cpp.
  • Revised paper (new section 5.4, three-model figures), refreshed HuggingFace demo (Qwen3-4B, three arms).
  • Offline pipeline and probe artifacts: https://github.com/Anyesh/j-space

Pre-workspace state is frozen as v1.1.