Help test 0.24: opt-in semantic search and background daemon

[luca-ctx]

The ctx CLI 0.24 release introduces two important opt-in features: semantic search and a background daemon.

Prior to 0.24, ctx search was lexical only, and incremental refresh generally happened in-process, usually right before ctx search, so results would be fresh.

Semantic indexing is much heavier than lexical indexing. Initial ingestion and incremental refresh take more CPU, memory, and wall clock time, so we do not want ordinary ctx search calls to be delayed by updating semantic indexes.

To make this feasible locally, 0.24 introduces a background daemon. When enabled, ctx setup and ctx import can start daemon-owned maintenance in the background. The daemon refreshes discovered local history sources, keeps lexical indexes fresh, and, when semantic search is also enabled, builds the local semantic sidecar.

When semantic search is enabled, the default search backend becomes hybrid: lexical + semantic evidence. You can still force the old path with ctx search --backend lexical.

Because this introduces new local-process and resource-use risk, it is currently behind explicit config opt-in. ctx does not write these defaults into your config file automatically.

Enable background daemon only:

# ~/.ctx/config.toml
[daemon]
enabled = true

Enable background daemon + semantic search:

# ~/.ctx/config.toml
[daemon]
enabled = true

[search]
semantic = true

After changing config, run:

ctx setup
ctx status

In 0.24, local semantic search is supported on Linux GNU and macOS arm64 builds. Other platforms should continue to work as lexical-safe builds. We will be following up to add support on the other platform builds in 0.25 or another upcoming release.

If you are interested in improving your search results and helping us harden this feature, please enable it and let me know how it goes: setup time, memory/CPU, indexing progress, incremental refresh behavior, search speed, and whether the results are actually better than lexical search for your real agent history.

CC'ing our contributors, thank you so much for your PRs so far. @atharva-patill @daisuke8000 @j2h4u @jongko54 @kj-9 @kuwa2005 @msokh @mtgibbs @nikrich @rNoz @robbe1912 @shinhyo @sin5ddd @yarikoptic @ZhiXiao-Lin

[j2h4u]

Hey @luca-ctx, thanks for a mention. What's the embedder model, is it multilingual?

---

Tbh I've been prototyping for a few days -- ingesting human replies from ctx and classifying them with tags from the SDLC taxonomy. Gonna make SDLC watchdog of a kind

[luca-ctx]

Yeah I think that wasn't the best decision. I have been doing some work on supporting Japanese and Korean in the lexical search, so I realized after shipping that we needed a different embedding model as well.

New version that I am working on is to use multilingual E5-small which I think might be a good balance of size and coverage.

One thing I'm really worried about is the CPU utilization while the semantic indexing is going on.

And btw I love the idea of the SDLC watchdog. We have a cloud service in private beta where we are working on something similar where you can analyze your team's agent's productivity. Would love to have you as a design partner on it if you're interested in trying it.

[j2h4u]

Yeah, it would be interesting to try. I love improving SDLC for the agents and humans

[j2h4u]

DX, AX, delivery, all of that

[luca-ctx]

0.25, just released, includes the updated embedding model, by the way

[j2h4u]

Yeah, that's slay! Pls check your LinkedIn, there's an invite from Max (hi hello it's me)