feat(core): add optional cross-encoder reranker stage to search

[ironhalik]

Hi,

I'm happily using basic-memory in my day to day workload, and wanted to chip in with an improvement - reranker.
The implementation was done by Opus 4.8, plus a review by Fable and GPT5.6 Terra.
The adversarial-review skill from the repo was also used.

I didn't do the full LoCoMo benchmark because I'm short on hardware right now. My homelab box offers ~6GB of memory bandwidth :)

I've been running this branch for couple of days now, and I'm pretty happy with the results.

Summary

Adds an optional cross-encoder reranker stage to vector/hybrid search. After
retrieval, the top candidates are rescored by a cross-encoder that reads the query
and document together, recovering relevant results that bi-encoder/FTS ranking
left just below the top-k cutoff.

Closes #950. Closes #618.

Reranking is off by default — existing users see zero change until they opt in.
When enabled it defaults to a local fastembed ONNX cross-encoder (no API, no
network), with an optional litellm provider for hosted rerankers (Cohere/Jina/
Voyage/Bedrock/self-hosted OpenAI-compatible endpoints).

Motivation

Per #950, roughly half of LoCoMo retrieval misses are ranking failures rather
than recall failures — the gold document is in the candidate set, just below the
cutoff — and there's ~20× latency headroom to spend on fixing it. A cross-encoder
is the standard fix: it re-scores a small candidate pool with full query↔document
cross-attention instead of a dot product.

What it does

• New RerankProvider protocol mirroring EmbeddingProvider, with two impls:
  • fastembed (default) — local ONNX cross-encoder, shares the embedding
    model cache, squashes raw logits to [0, 1] via sigmoid.
  • litellm — routes to any litellm-supported rerank API via provider/model
    strings (e.g. cohere/rerank-v3.5).
• Wired into the shared search_repository_base for vector and hybrid
  retrieval only. Explicit text/title/permalink searches keep user-requested
  ranking semantics and never rerank.
• Cross-project search (search_all_projects): each project reranks
  independently, so per-project scores aren't comparable across projects. A single
  merged rerank over the best candidates from the pooled results restores one
  calibrated global ordering.

Design decisions

• Off by default. A cross-encoder adds latency and a first-run model download;
  the opt-in keeps the default install unchanged. create_rerank_provider returns
  None when disabled, so the hot path stays allocation-free.
• Default model jinaai/jina-reranker-v1-tiny-en. Benchmarks higher recall@5
  than ms-marco-MiniLM at equal-or-lower latency on LoCoMo. (litellm can't route
  this local model name, so the litellm provider requires an explicit provider/model
  override — enforced by a config validator that fails fast.)
• One bounded rerank pool. Only the top reranker_candidates (default 20) are
  rescored; deeper pages skip the provider entirely (never rescore rows we won't
  return — matters for paid APIs).
• Monotonic score column. Reranked candidates carry [0, 1] relevance while the
  un-reranked tail still holds raw retrieval scores (up to ~1.3 fused). The tail is
  demoted strictly below the reranked floor so a tail row can't numerically outrank a
  reranked one — one comparable ordering without a second provider call.
• Fail-fast vs degrade. Transient faults (timeout, provider blip) degrade to
  retrieval order — an opt-in enhancement must never turn good results into a search
  outage. Permanent faults (missing deps, a provider that breaks its response
  contract — wrong score count, out-of-range/duplicate indices) surface loudly
  (RerankProviderContractError → 502) rather than silently returning un-reranked
  results with no signal.
• Process-wide provider singleton keyed on the fields that change provider
  identity (provider, model, litellm routing, resolved cache dir) so the model loads
  once. The cache dir is part of the key to avoid the FastEmbed cache defaults to /tmp/fastembed_cache — breaks MCP in sandboxed runtimes (Codex, ephemeral /tmp) #741/FastEmbed/ONNX embedding model is loaded multiple times per process, leaking ~25 GB of native memory #872 class of wrong-cache
  bug the embedding factory already guards against.

Configuration

All under BasicMemoryConfig, off by default:

key	default	notes
reranker_enabled	false	requires semantic_search_enabled
reranker_provider	fastembed	or litellm
reranker_model	jinaai/jina-reranker-v1-tiny-en	provider/model for litellm
reranker_candidates	20	top candidates rescored
reranker_max_document_chars	0 (full)	cap to bound latency on long notes
reranker_api_base / reranker_api_key	null	litellm only

Benchmarks

No gold-labeled recall benchmark (LoCoMo) here — the test box can't run it end to
end. Instead, a latency + ranking-quality measurement on a real 100-note
ops corpus, hand-labeled (n=20, one correct note per query). Directional, not
statistically significant, but every change traces to a real query.

Setup: fastembed + jinaai/jina-reranker-v1-tiny-en, candidates=13,
max_document_chars=1000, hybrid search. Same query set run twice, only
reranker_enabled flipped.

Ranking quality (gold = the single correct note, matched by permalink):

metric	off	on
recall@5	0.60	0.75
MRR	0.438	0.642 (+46%)
per-query wins / losses / ties	—	8 / 0 / 12

Zero regressions — the cross-encoder never demoted a correct answer. It promoted
buried-but-retrieved notes to the top (e.g. audit-log-shipping 8→1, restic→PBS
migration 4→1, fs-freeze experiment 6→1). The remaining ties are queries where
the gold note wasn't in the retrieval pool at all — a recall-stage ceiling the
reranker can't fix by design.

Testing

New coverage across the pipeline, providers, factory, and cross-project merge:

• Provider factory: singleton caching, cache-dir isolation, concurrent-race,
  unsupported-provider, disabled→None.
• Config validation: reranker-without-semantic rejected; litellm-with-default-model
  rejected; explicit litellm model accepted.
• Pipeline: reorder/rescore/tail-demotion, deep-page skip, graceful degradation on
  transient error, contract-error propagation, misaligned-score-count raises,
  hybrid reranks exactly once.
• Cross-project merge: rescore pool, no-op when disabled/single-result, tail
  demotion, transient degrade, permanent-fault surface, non-semantic search types
  skipped.

just check + targeted pytest green.

[CLAassistant]

All committers have signed the CLA.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 63ef501943

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

[chatgpt-codex-connector]

Reject duplicate rerank result indexes

When LiteLLM returns a malformed results list that repeats an index while still including every expected index (for example [0, 1, 0] for two documents), this loop overwrites the earlier score and seen still equals the full expected set, so the provider contract break is accepted and search is reranked with arbitrary duplicate data. The surrounding contract says each index must appear exactly once; fail fast when index in seen before assigning the score.

Useful? React with 👍 / 👎.

[ironhalik]

Good catch — confirmed and fixed in e2cd4ac.

[0, 1, 0] for two documents did slip through: the range check passed each item, the duplicate index overwrote its own score (last-write-wins), and seen == {0, 1} still satisfied the coverage check, so the contract break was accepted. Added an explicit if index in seen: raise RerankProviderContractError(...) before assigning, plus a test_duplicate_index_raises regression test alongside the existing out-of-range / incomplete-coverage cases.