kinfer v0.1.9

A model with experts in host memory keeps its prefix pool when memory is tight, so a conversation's next turn is no longer read whole again.

curl -fsSL https://raw.githubusercontent.com/LocalKinAI/kinfer/main/scripts/install.sh | sh

• The prefix pool stays when memory is tight. A conversation's next turn reuses the one before it through the prefix pool. A pool needs llama.cpp's KV cache unified, and a pass over a unified cache is sized for all of it. On the box, Flash-Next's four slots of 32768 took a pass of 5.8 GiB, against 2.2 for the conversations' own caches. Placement had already given that memory to the expert cache. So when the box's other services held memory, the pool was measured out, and every turn of a conversation was read whole again. Now the expert cache placement chose gives up as much as half of itself before the pool loses an entry.

Flash-Next on an M3 Ultra, -slots 4, with 26 GiB held by another process	v0.1.8	v0.1.9
prefix pool	none	one entry, with 2.5–3.1 GiB less expert cache
turns 2 and 3 of a 7,800-token conversation	12.6–13.1 s	1.0–1.2 s
decode, a 113–116-token reply	24.2–25.1 tok/s	21.3–21.8 tok/s
the three-turn conversation	47 s	26 s

With a 45,000-token agent prompt, that saves about 70 s on each turn.

• A cache set with -moe-cache stays as given.
• Where the pool already fit, nothing changes: on a MacBook Pro, Flash-Next and Ornith-397B load exactly as they did.

The patched libraries are unchanged (darwin_arm64_b11998k5). CHANGELOG.md

kinfer-v0.1.9-darwin-arm64.tar.gz sha256 a0e6b15c5afeea71f1fa8200fdce368dbf6f7a8515f48f4a56efd8bc16f858ba