Search accuracy, and the project's first license.
Search now answers the question you asked
Searching a word the journal had never recorded used to return a screenful of
unrelated moments. Two causes, both in hybridSearch:
- The nearest-neighbour query had no distance ceiling.
ORDER BY embedding <=> query LIMIT 20always returns twenty rows however far away they are —
there is no such thing as "no semantic results", only the least-bad twenty. - Reciprocal-rank fusion then scored a rank-0 semantic guess (
1/60) above a
rank-1 literal word match (1/61), so the word you typed was not reliably
first.
/search is now literal. Results are segments that actually contain your
terms, ranked by ts_rank, each deep-linking to the exact second. If the log
never said the word, you get nothing rather than a confident-looking stranger.
An intermediate version of this release surfaced semantic neighbours as a
separate "similar topics" list. It was removed: measurement showed cosine
distance is not a reliable relevance signal here. On a 600-segment corpus a
keyboard mash landed nearer (0.34) than ordinary words like "food" (0.40), and
the safe threshold moves with corpus size — the same gibberish sits at ~0.47 on
a 90-segment corpus. A list that looks authoritative should not rest on that.
Meaning-based retrieval stays where a model has to justify each citation:
Ask and the live agent's search_journal tool still receive literal
matches plus bounded semantic neighbours, so natural-language questions keep
working. Both also benefit from the fix, since the unfiltered neighbours were
previously reaching the LLM as context.
License
The project is now licensed under the
PolyForm Noncommercial License 1.0.0.
Personal, hobby, research, educational, and nonprofit use is granted, including
modifying and redistributing. Commercial use is not. This is a source-available
license, deliberately not an OSI open source one.
Also in this release
SEARCH_SEMANTIC_MAX_DISTbounds semantic retrieval for Ask (default0.33)- Search page copy corrected: it no longer claims to be semantic, and the
Ollama-offline banner is gone from keyword search - ESLint now honours the leading-underscore convention the code already uses
- README: search and Ask sections, config table, and screenshot refreshed
Upgrading from 0.1.1
No schema change. Rebuild and restart:
(cd web && npm run build) && systemctl --user restart memento-webFull changelog: v0.1.1...v0.1.2