Skip to content

Krill v0.26.0

Choose a tag to compare

@srvsngh99 srvsngh99 released this 21 Oct 19:41
· 34 commits to main since this release

Embeddings on the Neural Engine, one setting away. New embed_device = "gpu" | "ane" | "auto" (also KRILL_EMBED_DEVICE and a per-request device field) picks where EmbeddingGemma 2 text embeddings run. On the Neural Engine a query takes about 4 ms instead of 14 ms, uses about 6 mJ instead of 80 mJ, and the server peaks at about 160 MB and rests near 50 MB, with vectors matching full precision (cosine 0.9993). Long inputs, images, audio and video go to the GPU automatically, and the full-precision embeddinggemma-2 build is never moved by the setting. Build the package once with make ane-package (see docs/ANE_EMBEDDER.md); the default stays gpu.

Fixed: krill serve now stops cleanly on SIGTERM/SIGINT (in-flight requests get up to 3 s) instead of ignoring them.

See CHANGELOG.md for details.