Skip to content

v1.0.1

Choose a tag to compare

View all tags
@github-actions github-actions released this 11 Jun 17:27
· 38 commits to main since this release
v1.0.1
7de1a0e

ken v1.0.1 — faster hybrid search

A patch release: same retrieval quality, ~3× faster hybrid search. No API changes — mcp.Run, mcp.NewServer, mcp.NewCache, FormatResults, the chunk.Chunker interface, and the MCP wire format are unchanged from 1.0.0, so this is a drop-in upgrade.

~3× faster hybrid search (measured)

The headline comes from a aikit bump (v1.0.0 → v1.4.0) that rewrote the semantic-arm cosine scan (ann.Flat.Query) from a scalar-float64 loop to a SIMD-float32 dot kernel with 8-vectors-per-pass streaming. Benchmarked before/after on the same machine:

  • Flat.Query 11.7× faster in isolation (2061 µs → 176 µs over 8 000×256).
  • End-to-end hybrid search p50 −66 % — 4.58 ms → 1.56 ms on a ~13 k-chunk corpus. The scan is O(N) in chunks, so the win grows with corpus size (it's invisible on tiny repos where search is already sub-millisecond, large on real ones).
  • Indexing time unchanged — the bump didn't touch the index path.

Quality is provably untouched: recall@10 was re-verified identical (0.967 NL / 0.995 symbol), and embedding parity holds (golden cosine ≥ 1 − 1e-5 vs the Python reference). The neural reranker is unchanged too (the encoder vectorization in this aikit range targets the int8 path; ken's default reranker runs f32).

Also in this release

  • Deserializer fuzzing. FuzzDeserializeIndex and FuzzDecodeRerankCache validate the hand-written adversarial-input defenses in the KEN1 / KNRC binary parsers — the index parser is real attack surface, since ken-mcp auto-loads <repo>/.ken/index.bin from shallow-cloned remote repos. 2.6M executions, zero crashers.
  • Model-download fix. ken download-model no longer mistakes a leftover Git-LFS / HF-hub pointer stub (or a broken symlink / truncated file) for a real model. The "already present" check now validates file size, so a stubbed model gets re-downloaded instead of silently failing to load later.
  • Robustness from the aikit bump on exactly the paths ken runs: a safetensors mmap-lifetime guardrail and two fuzz-fixed untrusted-tensor crashes in embed, plus bm25 / chunk indexing-pipeline fuzzing.
  • Docs accuracy sweep. A full pass aligned the documentation with the post-1.0 / post-extraction reality (package locations, the 10-tool MCP surface, 13 languages, the mcp.Run library example, regenerated third-party licenses, and a current-state ARCHITECTURE.md).

Upgrade

brew upgrade --cask ken      # or: scoop update ken

Or grab a binary below, or go install github.com/townsendmerino/ken/cmd/ken-mcp@v1.0.1. Full detail in CHANGELOG.md; benchmark methodology in docs/PERF-expectations.md.

Assets 9
Loading