lilbee: a single-executable local search engine you can talk to. It runs the models, crawls the web into your library, and reads 90+ document formats with Kreuzberg

[tobocop2]

Hey folks,

For the last few months I've been working on lilbee: https://lilbee.sh and https://obsidian.lilbee.sh/

At its core, lilbee is the whole local-AI stack in a single executable. It runs the models, indexes everything you own (90+ file formats and 150+ programming languages, thanks to Kreuzberg), crawls and saves web pages, and lets you search and talk to all of it in plain English, with every answer citing the exact file and line. There's no separate model server to stand up, no vector database, no containers to configure. It runs entirely on your own hardware and only touches the cloud if you point it at a cloud model.

The core components are llama.cpp (runs the models), Kreuzberg (reads the documents), and LanceDB (the vector store), all bundled into the one binary.

You can use it however you want. It exposes an MCP server so any coding agent can search your stuff and get back cited snippets, and an HTTP server that GUIs build on (the Obsidian plugin is one). There's also a full-screen terminal app, a CLI, and a Python library. The TUI is pretty damn cool in my opinion and it's primarily how I talk to my stuff when I don't need a GUI. I use the Obsidian plugin for that need. Direct opencode integration with local models is coming soon.

Here is the kind of agent integration that gets me excited: a coding agent (opencode, with a cloud model) writes a procedural Godot level generator, and every Godot API call is cited back to the class reference that lilbee indexed. A side-by-side benchmark measured 4 hallucinated APIs without lilbee and 0 with.

It also has its own model manager built on llama.cpp, so one program browses Hugging Face, pulls models, and runs them on Metal, Vulkan, or CUDA. Already on Ollama or LM Studio? Point lilbee at them and keep using them.

Tutorial reels: https://lilbee.sh/tutorial/ and https://obsidian.lilbee.sh/tutorial/
Obsidian plugin is now in the community store: https://community.obsidian.md/plugins/lilbee

Early on, my vision was to just talk to websites and my documents privately, essentially building an Encarta 99 out of my own stuff that I could talk to, and that's still the core of my mission.

I'm trying to get into horticulture at scale, so my earliest use case of lilbee was talking to horticulture forums that I crawled and indexed into lilbee. I was blown away by how useful my hardware suddenly became to me. For the last few years my laptop was just a glorified web browser. I didn't use it for games at all, and I last heard the GPU fans 5 years ago when I was doing ML work.

I wound up engaging all of my closest friends and family and soliciting lilbee to them against their wishes, and now I have helped several of my friends with their jobs or hobbies:

• One of my friends is a biomedical engineer and he uses lilbee nearly daily, ranging from worm research to Star Wars X-Wing and Myst strategy guides. He got further in Myst in the last few months than he ever did in his life by talking to the strategy guide.
• Another of my friends is a PhD chemist and has to operate some crazy microscopes that have Python SDKs. She used lilbee indexed with the microscope SDK manual with DeepSeek to produce a fully functional script that she could not have ever conjured on her own. This was completely offline using a PDF manual.
• Another one of my friends got into building games. He was doing it with free tier cloud AIs and was getting hallucinated Godot APIs throughout all his stuff. He wasn't a programmer before and he struggled to circumvent this. I gave him lilbee on top of opencode, and using the free tier of MiniMax 2.5, he produced fully working game prototypes.

In the earliest lilbee incarnation, I essentially reinvented a subset of what Kreuzberg is. I was thankfully already using tree-sitter-language-pack (thank you!!), but for some odd reason I didn't find Kreuzberg. I reached out to Goldziher and shared what I was working on specifically to thank him for tree-sitter-language-pack. He pointed me to Kreuzberg, and within a few weeks I was contributing all of my discoveries to Kreuzberg and became a member of the open source team!

Stuff I contributed that I'm pretty proud of:

• improved or introduced chunking for the RAG pipelines for: csv, xml, markdown, yaml
• introduced support for a new format: jsonl
• introduced semantic chunking
• exposed PDF rasterisation for externally managed vision models

On top of making my processing layer way faster, Kreuzberg enabled me to drop 9+ dependencies. The document processing functionality is the core of what makes lilbee awesome, and Kreuzberg is incredible. What's amusing to me is that Goldziher and I briefly worked together years ago, and the stars just happened to align because we are now solving similar problems together in our free time.

Anyway, lilbee is at a place I'm now comfortable sharing, so I'm posting it in the communities whose projects it's built on, starting here. I'm holding the broader launch until my next big feature lands:

I'm moving the project from an in process inference/embedding architecture to something scalable, using llama-server and llama-swap. This will enable lilbee to scale from a single laptop to a multi GPU beast. I've recently verified MiniMax M2 split across 3 graphics cards, all managed by lilbee. This is a big deal in my opinion because it's a single executable that has a search engine, model manager, and a web crawler. Multi-GPU work (and the opencode integration) is here: tobocop2/lilbee#267

Early on I chose an in process approach because I wasn't thinking big. I wanted something optimized with no HTTP overhead, no servers to manage, and something I could easily invoke in the terminal on demand. This let me scale to my laptop fine, but now I'm moving away from this architecture because I'm solving much bigger problems and I need to scale across multiple GPUs.

The single executable approach is something I'm excited about, and I feel the local AI space is too complicated for the layman to onboard with. I've been very active in the Obsidian forum and I wound up making a new friend because of my approach. He confessed to me that he is not technical at all but wanted local RAG. For him to try even the most widely popular projects is daunting. Having to install multiple tools and configure them is hard, especially if there's containers and networking involved. Having the local models and search engine right in Obsidian enables people to use a battle tested GUI with a huge community, and it's appealing for people who don't want to install a new app for this specific purpose. My new friend was talking to his solar panel docs within minutes and he was blown away. This was his first successful experience with local AI. He never opened up a terminal even in his life, so the idea was just daunting. From Obsidian to the community plugin store to lilbee to the setup wizard to talking to his stuff. That's local AI democratization in my opinion. It becomes less for nerds once it becomes easier to use.

The general local AI hosting patterns aren't friendly and they're just annoying to manage, and that's why I rolled a model manager into lilbee. I would have built it on top of Ollama or LM Studio if they could have been used as libraries, so with some reservations I built my own, and it's been awesome. lilbee is backwards compatible with Ollama and LM Studio because I know people want to use battle tested tools. However, lilbee's model manager is a first class part of the tool.