Skip to content

Krill v0.27.0

Choose a tag to compare

@srvsngh99 srvsngh99 released this 21 Oct 22:02
· 21 commits to main since this release

Faster embeddings, local logs, and upgrades that apply themselves. Most embedders and the bge-reranker-base reranker can now run on the Neural Engine with the same embed_device setting. For bge-base a query drops from 8.9 to 2.6 ms and the server's peak memory from about 1 GB to 34 MB. Krill now keeps logs on your Mac and nowhere else: krill logs, krill doctor and krill bug-report read them, and nothing is ever sent anywhere. After brew upgrade, a running service switches to the new version by itself once it is idle. Upgrading from 0.26.x to 0.27.0 needs one manual service restart, because the old version has no watcher.

Details

  • Neural Engine packages are built locally from the weights you already pulled with make ane-package MODEL=<name>. Each package must match the GPU results (cosine 0.999 or better for embedders) before it installs. Models without a package stay on the GPU.
  • Measured speedups run from 1.7x to 4.4x at p50, with process peak memory 8x to 32x lower.
  • The default stays embed_device = "gpu". The same setting also routes /v1/rerank.
  • A daily event log, agent session transcripts, crash notes and retention limits all live under ~/.krill. The event log holds no prompt or reply text. Session transcripts keep the conversation on your Mac; set session_log = "meta" or "off" to drop it.
  • krill bug-report makes a redacted zip and a prefilled issue link. It sends nothing.
  • krill serve restarts onto a new version only under launchd and only when idle. Turn it off with auto_restart_on_upgrade = false.
  • /metrics and /v1/status now report the server's own uptime and real memory use, plus request, error and latency counters.
  • The README gained a Privacy section.