v0.6.90b420.dev730
The headline of this release: lilbee now scales ingest across every GPU in the box. Millions of files land in hours on datacenter hardware.
Ingest across every GPU
One process used to feed all your cards, and its serial plan-extract-write stage starved them; on one measured run, eight GPUs ingested slower than four.
lilbee sync now fans out on its own: one worker per card, each with its own store, engine slot and slice of the files, merged into one index at the end. Nothing to configure, one progress bar. Small jobs stay single-process as before.
Measured on MS MARCO, 8.8 million passages, 8B embedder, 4xH100: 103 docs/s single-process, around 200 docs/s with the fan-out. The full 8.8M lands in about half a day on one machine.
Big runs are also harder to kill: an add finishes when some files skip and only fails when nothing was indexed, and an empty GPU probe retries before failing placement.
Concept wiki
lilbee can now generate a wiki from your documents: cited concept and entity pages, so you can browse what your knowledge base knows instead of only querying it. Pages carry citations back to the source chunks.
Retrieval quality, with receipts
Reranking now applies in search() itself, not just the ask and chat paths, so plain search results get the same quality boost. Vector search probes more IVF partitions at scale, recovering recall that large indexes were quietly losing.
Both improvements are backed by a new MS MARCO passage retrieval benchmark in the repo. The full reranked pipeline scores nDCG@10 of 0.437, recovering essentially the embedder's ceiling, and the dense retriever alone lands MRR@10 of 0.346, competitive with well-known published baselines.
Cancel the slow stuff
Long operations now stop cleanly when you cancel them. Ctrl-C interrupts a CLI wiki run at a boundary, and wiki builds, lint and prune passes, dataset exports, model pulls and dataset imports all abort promptly when the caller cancels instead of grinding on in the background.
Engine
llama-server now sizes its thread count to the container's CPU quota instead of the host's core count, so a pod that advertises 96 cores but grants 8 no longer drowns itself in threads.
MCP server
The MCP server migrated to the mcp 2.x SDK, keeping lilbee current with the protocol ecosystem agents actually use.
Models and site
The featured model list is now live HuggingFace picks instead of a hand-curated snapshot. The site's sticky nav behaves on mobile browsers again.
Dependencies
filelock moved to 3.32.2 after unblocking the upgrade, the GitHub Actions toolchain took its grouped bumps, and the litellm cap is now documented and tested against the real constraint that holds it in place.