v0.6.90b420

This release is the culmination of every dev release from dev710 through dev734. That work is now largely stable, so it is promoted here as one release.

It is the biggest release lilbee has shipped. Since 0.6.66, the model engine was rebuilt from the ground up, and everything above it grew to match: search, ingest, extraction, the TUI, and the agent integrations. Almost all of it follows from one decision. Models moved out of lilbee's process and into a managed fleet of their own servers.

Models run as their own servers

Models used to run in one process with the app. A native crash took everything down, and heavy embedding work competed with your chat.

Now chat, embeddings, reranking, and vision OCR each run on their own managed server. lilbee measures each model's real memory footprint before launch and plans what fits where. Models that never run at the same time make room for each other on demand. With enough VRAM, everything stays loaded.

Every lilbee process on the machine shares one engine instead of loading its own copy of the weights. Servers are bound to lilbee's lifetime, so no exit, crash included, can leave a model running or VRAM pinned. The engine ships prebuilt. There is nothing to compile.

Built for scale

• One model can use all of your GPUs. A model bigger than one card splits across several, sized against each card's real memory. Estimates advise, the load decides, and only physics refuses a model.
• Ingest fans out on its own. One worker per card, merged into one index at the end, nothing to configure. 8.8 million passages land in about half a day on one machine.
• The bottlenecks are gone. Embedding starts on the first batch instead of after a full plan, embedding decode is 7 times faster, and an ask that spent 86 seconds reranking now spends 4.6.

Search was rebuilt

Search, ask, the TUI, and the MCP tools now share one relevance policy, so a query comes back ordered the same way everywhere. Every question runs through two arms, one for meaning and one for the exact words, fused into a single bounded score that the whole pipeline agrees on.

The question shapes that used to fail now work. "Summarize report_482.pdf" resolves to that document, in page order. "How many documents mention X?" gets an exact count, not a hedge. A follow-up question is resolved into a standalone query first. Citations line up one-to-one with the sources, and you can click one to open the document.

The numbers are measured on the MS MARCO passage benchmark, in the repo: MRR@10 of 0.346 for dense retrieval alone, nDCG@10 of 0.437 for the full reranked pipeline, essentially the embedder's ceiling.

A wiki grown from your documents

lilbee can generate a wiki from your library: cited concept and entity pages, so you can browse what your knowledge base knows instead of only querying it. Pages link to related subjects, and a page written on request joins the graph like any other.

It runs on the GPU you have

NVIDIA, AMD, Intel, and Apple silicon all work, down to consumer integrated graphics, for placement, live utilization, and memory sizing. The CUDA and ROCm builds bundle their runtimes, so the driver is all you need. ROCm is a real HIP backend now, validated on datacenter and consumer cards. The x86 builds pick the best CPU variant at startup, so a modern CPU keeps its full speed while a 2008 machine still runs.

Fast to first chat

lilbee opens straight to chat in a few seconds instead of making you wait on the model. The first answer of a session loads the model live inside the answer bubble. A relaunch with a warm engine reaches a usable chat in about 5 seconds. Model downloads run about 14 times faster on a cold pull.

Your coding agent runs on lilbee

lilbee launch claude, lilbee launch opencode, and lilbee launch hermes set lilbee up as the model provider inside the agent's own config, register your library as a search tool, and drop you into a working session. Claude Code gets the Anthropic Messages API served locally, with streaming, tool calls, and thinking blocks. Any client can fetch a paste-ready config from the server, so nothing goes stale.

Extraction handles anything

Document extraction covers more than 100 formats, including legacy Office, ebooks, email, and archives, with 100+ OCR languages. Scanned pages OCR through your own vision model. Tables and page layout come out structured, and titles and authors survive ingest. A full run over 369,758 documents finished with zero extraction failures at 14.2 ms per page.

The TUI improved

• A fresh install lands in the catalog with fit chips for your hardware, and the first completed download becomes the active model. No wizard.
• One key vocabulary across every screen. q and Escape go back, m opens the catalog, ? opens help. The footer fits an 80-column terminal.
• Catalog search queries HuggingFace as you type.
• Colors are correct on 256-color terminals, over SSH, and under tmux.
• The placement editor (Ctrl+G) shows live per-GPU utilization: pin roles to cards, preview the fit, apply.
• Switching models mid-answer keeps the answer, and /cancel works during a stream.

It keeps running

Each server has a health check and restarts on its own if it dies, while the app and your other models keep working. Transient errors are retried instead of dropped. A prompt after an idle unload reloads the engine instead of erroring forever. Failures say what happened: a missing engine names the install command for your hardware, a missing model is named, and a model whose context would collapse is refused up front, with the numbers.

Your data is yours, and stays consistent

Index writes are atomic, so a crash cannot leave the index half-written. A removed document stays removed, and adding it back works. Files are indexed where they live, with no second copy. Every HTTP route requires the session token. Nothing downloads without being asked, and ask prints the answer alone on stdout, so it pipes cleanly.

Chat sessions

Conversations save as you chat and survive quitting. List, resume, rename, and delete them from the TUI, the HTTP server, MCP, and the CLI. Agent sessions stay out of your human lists unless you claim them.

Every way you might install it

Homebrew, AUR, Nix, Docker, Flatpak, Snap, Scoop, and standalone binaries, in CPU, CUDA, ROCm, and Vulkan variants where the platform has them, plus compat builds back to 2008-era CPUs and macOS 12 Intel Macs. Scoop reads your NVIDIA driver and picks the right CUDA build. For a Python you manage, pip and uv install lilbee[engine] from a per-hardware index. Every release ships its openapi.json, so clients pin a schema instead of guessing.