Repository navigation
v0.48.2.0
Your search reranker now runs on Voyage, and every surface tells you whether it is actually running.
The cross-encoder reranker that balanced and tokenmax search modes run
after fusion used to default to a ZeroEntropy model whose hosted API shuts
down on 2026-09-04. This release moves the default to Voyage rerank-2.5
ahead of that date. It rides the same VOYAGE_API_KEY as the embedding
default, so a brain that already embeds with Voyage reranks with no extra
setup. A brain without the key keeps working: search falls through in fusion
order, quietly, and gbrain search modes, gbrain search --explain and
gbrain doctor all say so instead of leaving you to guess why results look
unranked.
Say to your agent: "check whether my brain's reranker is actually
running" — "turn reranking off for now" — your agent runs
gbrain search modes / gbrain doctor, then either exports VOYAGE_API_KEY
or runs gbrain config set search.reranker.enabled false.
How to use it
gbrain search modes # Reranker: voyage:rerank-2.5 (enabled) — VOYAGE_API_KEY present
gbrain doctor # reranker_health names the fix when the key is missing
gbrain search "acme-example series A" --explain # shows `degraded: reranker_skipped (no_key)` if it was skippedNo Voyage key and no wish to rerank: gbrain config set search.reranker.enabled false.
Things to watch
- Every cached search result set is re-keyed once by this flip (the reranker
model is part of the cache key in all three modes). One cold pass, then the
cache refills withincache.ttl_seconds(default 3600). - A brain that explicitly configured a ZeroEntropy
zerank-*reranker keeps
it until 2026-09-04; from that date the dead hosted call is skipped before
any HTTP, with one audit row per process and a single stderr line naming
the switch command. - The LongMemEval reranker on-vs-off rows landed in this release (Measured,
below; the reranker-off row is a reproduction of v0.48.0.0, not a new
result). The rest of the paired rerank A/B (the autocut floor, and
NamedThingBench, cat13b and world-v1) is pre-registered in the ranker wave
that follows this release and publishes there, wins and losses. - If you run
tokenmax, expect worse small-k recall thanbalancedfor now.
tokenmaxturns on LLM multi-query expansion, and on LongMemEval at k=5
expansion drops strictrecall_all@5from 93.19% to 54.89% (258/470,
2026-09-02, v0.48.2.0; paired +3 / -183 against plain hybrid). The
mechanism is equal-weight variant vector lists in RRF, and the fix is
tracked in the TODOS.md entry on master, "multi-query expansion dilutes
small-k retrieval now that fusion is clean" (weight variant lists below the
original, cap their contribution, or expand only when the original query's
evidence is weak). Until it lands,gbrain config set search.mode balanced
keeps the reranker and drops expansion.
Measured
Five retrieval arms on the public LongMemEval benchmark, run at this release
so the reranker flip is judged against a reranker-off baseline from the same
day. Metric: LongMemEval's official session-level recall_all@5 (every gold
session inside the top-5 distinct retrieved sessions; retrieval only, no
reader model). Dataset: longmemeval_s, cleaned September 2025 revision,
500 questions, 30 abstention questions excluded, 470 scored. Run on
2026-09-02 at gbrain v0.48.2.0 (commit 172df27, PR #4792), search mode
balanced pinned, autocut OFF, reranker OFF except the two rerank arms
(which run voyage:rerank-2.5, this release's default), embedder
openai:text-embedding-3-large at 1536 dims, k=5, single run, 0 errors in
every arm. "Paired" is per-question against the hybrid row: questions this
arm gets right that hybrid missed / questions it loses that hybrid had.
| Arm | recall_all@5 | recall_any@5 | Paired vs hybrid | Notes |
|---|---|---|---|---|
| hybrid (reranker off; the like-for-like row) | 93.19% (438/470) | 98.72% | +0 / -0 | nDCG_any@5 93.32%; distinct sessions in the top 5: 5 on 422 questions, 4 on 47, 3 on 1 (mean 4.90); p50 3.7 s / p99 6.3 s per question |
| hybrid + LLM multi-query expansion | 54.89% (258/470) | 86.60% | +3 / -183 | nDCG_any@5 71.68%; mean 5.00 distinct sessions; p50 5.1 s / p99 8.0 s; the v0.48.0.0 receipt had 49.6%, same verdict: harmful at small k |
| hybrid, session-diverse (3x over-fetch, top-5 distinct sessions) | 93.40% (439/470) | 98.72% | +1 / -0 | nDCG_any@5 93.38%; mean 5.00 distinct sessions; p50 3.7 s / p99 6.4 s |
hybrid + rerank (voyage:rerank-2.5, this release's default; the release default path) |
95.32% (448/470) | 99.79% | +18 / -8 | nDCG_any@5 95.77%; mean 4.89 distinct sessions; p50 3.8 s / p99 6.3 s |
| hybrid, session-diverse + rerank | 95.53% (449/470) | 99.79% | +19 / -8 | nDCG_any@5 95.82%; mean 5.00 distinct sessions; p50 3.8 s / p99 6.3 s |
By question type (recall_all@5, hybrid reranker off, then hybrid + rerank):
knowledge-update 98.6% (71/72), then 100% (72/72); multi-session 92.6%
(112/121), unchanged at 92.6% (112/121); single-session-assistant 100%
(56/56), unchanged; single-session-preference 96.7% (29/30), then 100%
(30/30); single-session-user 98.4% (63/64), then 100% (64/64);
temporal-reasoning 84.3% (107/127), then 89.8% (114/127). The expansion arm
loses in every type (multi-session 34.7%, temporal-reasoning 40.2%,
knowledge-update 62.5%, single-session-user 78.1%, single-session-preference
80.0%, single-session-assistant 82.1%). The full per-type table is in
docs/eval-bench.md.
- Identical to v0.48.0.0. 438/470 matches the v0.48.0.0 receipt (PR
#4787) exactly, per type as well as in total: the reranker succession does
not change reranker-off retrieval. - The reranker earns +2.13 points on the release default path. With
voyage:rerank-2.5on, hybrid moves from 93.19% to 95.32% (+18 / -8
paired); every gain outside multi-session, where the reranker changes
nothing (112/121 both ways), and the largest in temporal-reasoning (107 to
114 of 127). Any-hit rises to 99.79%: the reranker is promoting sessions
that were already in the candidate pool, not finding new ones. - Expansion is harmful at k=5, plainly.
tokenmax's LLM multi-query
expansion drops strict recall from 93.19% to 54.89% (+3 / -183 paired) and
any-hit from 98.72% to 86.60%, on the same 470 questions, with zero
expansion errors. Equal-weight variant vector lists push correct sessions
out of the top 5. Nobody should read the reranker-on numbers as a
tokenmaxresult;tokenmaxwas not measured with the reranker here. - Slot starvation is not the miss class. Session-diverse over-fetch (3x,
keep the top-5 distinct sessions) fills every top-5 to 5.00 distinct
sessions (hybrid alone returned fewer than 5 on 48 of 470) and gains
exactly one question (93.19% to 93.40%, +1 / -0; 95.32% to 95.53% with the
reranker). The 22 questions the reranker still misses are ranking misses,
not duplicate sessions eating slots. - The bracket below the fix, disclosed. Between v0.28.8 (83.40%
recall_all@5, May 2026; the 97.66% any-hit figure that led the May report
is a diagnostic only) and v0.48.0.0, an unpublished keyword-fallback fusion
regression dropped hybrid to 51.3%. Re-measured on 2026-09-02 at the
v0.47.8.0 evals pin (2a56b51), same 470 questions: 51.39%recall_all@5.
v0.48.0.0 fixed it (93.19%). The ceiling at k=5 is 99.4%, because 3
questions carry 6 gold sessions; pure vector on the same corpus scored
93.8% (v0.48.0.0 receipt), so hybrid is roughly neutral on this benchmark. - Strict vs loose, in one line.
recall_all@5counts a question only when
every gold session is in the top 5;recall_any@5counts it when any one is.
The 98.72% any-hit number is the loose diagnostic and is the one that sits
beside most published "LongMemEval retrieval" scores; the 93.19% strict
number says nothing about answer accuracy (gbrain has published no
LLM-judged answer-accuracy run).
Receipts, per-question rows (rerun-2026-09-02-v0.48.2.0-*.ndjson, one per
arm, plus rerun-2026-09-02-v0.48.2.0-all-arms.json and the two SVG charts),
and the arm table:
gbrain-evals docs/benchmarks/2026-05-07-longmemeval-s.md.
Say to your agent: "Run the public LongMemEval benchmark against my
brain" — your agent runs
gbrain eval longmemeval <path-to-longmemeval_s_cleaned.json> --retrieval-only --top-k 5 --by-type --no-trajectory
(the CLI's --by-type summary is any-hit recall, and it runs the release
default, reranker on when VOYAGE_API_KEY is set; the strict recall_all@5
receipt comes from the gbrain-evals runner, which pins reranker and autocut
off: bash eval/runner/longmemeval-batch.sh --adapters hybrid --embedding-model openai:text-embedding-3-large --embedding-dims 1536).
To take advantage of v0.48.2.0
gbrain upgrade should do this automatically. If it didn't, or if gbrain doctor
warns about the reranker:
- Check what is running:
gbrain search modes - Either add the key or turn reranking off:
export VOYAGE_API_KEY=... # or: gbrain config set voyage_api_key ... gbrain config set search.reranker.enabled false # if you do not want reranking
- Verify:
gbrain doctorshowsreranker_health: ok. - If any step fails, file an issue at https://github.com/garrytan/gbrain/issues
with the output ofgbrain doctor.
Itemized changes
Changed
- Reranker default →
voyage:rerank-2.5in all three mode bundles.
DEFAULT_RERANKER_MODELinsrc/core/ai/defaults.tsis the one code home;
src/core/search/mode.tsandsrc/core/ai/gateway.tsimport it. The
retired ZeroEntropy value stays a named constant (LEGACY_DEFAULT_RERANKER_MODEL)
for the sunset row, so an explicitzeroentropyai:*config still
short-circuits past the date instead of hanging five seconds per query. - Keyless brains fail open quietly.
gateway.rerank()skips a reranker
whose provider key is absent before any HTTP call (RerankError('no_key')):
one audit row per process per model, no stderr.applyRerankerpasses the
results through and reports the skip, andhybridSearchstamps
{stage: 'reranker_skipped', reason: 'no_key' | 'sunset_short_circuit'}
onHybridSearchMeta.degraded;--explainrenders it via
formatDegradedSummary. The stamp does NOT shorten the query-cache TTL
(it is a configuration state, not a transient provider limp; the cached
rows carry the stamp so a hit stays honest).authnow means "key present
but rejected" (HTTP 401/403). - Init writes less.
writeNewInstallRerankerDefaultwrites no
search.reranker.modelrow when the default is ready (the bundle already
resolves to it); keyed installs WITHOUT a Voyage key still get
search.reranker.enabled false; keyless installs still get nothing. gbrain doctorreranker_healthresolves enablement and model through
resolveSearchMode(loadSearchModeConfig)and checks readiness before
reading the audit (through the same DB-merged config plane the CLI hands
the gateway, so DB-plane keys and self-hosted base-URL overrides count):
enabled-but-not-running warns with the paste-ready fix;no_key/
sunset_short_circuitskip rows are reported as information once the
reranker is ready; the auth, payload, budget, transient and unknown
ladders are unchanged.- Budget scopes: the rerank budget reservation is now taken only when an
HTTP call will actually be made (and before the abort timer is armed), so
skipped rerankers under a cost cap no longer accumulate unsettled
projections; the estimate honors the provider's tokenizer density (Voyage
counts roughly one token per character on dense text), so a cap cannot be
under-reserved by dense CJK or JSON documents. - Doctor
search_mode: an explicitsearch.reranker.modelrow that equals
the mode bundle's own default (every Voyage install since v0.46.3 has one) is
called redundant with a precisegbrain config unset search.reranker.model
instead of a--resetthat would also wipe your other tuned search knobs. - Doctor never blames a missing reranker key on a brain that has no embedding
provider (search runs keyword-only there and the reranker never fires), and
audit rows for a retired model no longer make the live default warn. - Empty-result copy: a skipped reranker is a ranking-only state, so the MCP
empty-result block and the CLINo results.line keep saying "clean miss"
instead of reporting a retrieval degradation. - Remote MCP callers (
search_modes) get the reranker verdict without the
host's provider-key inventory or self-host topology; the local CLI dashboard
keeps the full readiness line. - Init: a ZeroEntropy embedding pick is treated like any other keyed
non-Voyage install (search.reranker.enabled falsewithout a Voyage key), and
a--forcere-init sees a Voyage key that lives only in the brain's config
table. - ZeroEntropy detection everywhere (
rerankerSunset, the upgrade banner,
the doctor checks, init) matches the provider id case-insensitively and in the
provider/modelform, the same way the gateway resolves it.
Added
src/core/ai/reranker-readiness.ts—rerankerReadiness(model, env, now?)describeRerankerFix(r), the one readiness predicate behind
gbrain search modes, doctor and init; pinned to agree with
isAvailable('reranker', m)on an env × model matrix.
gbrain search modesattributes the fivereranker_*knobs like every
other knob, prints a per-bundlereranker=… topNIn=… autocut=…line and a
runtimeReranker:readiness line;--jsoncarriesreranker_readiness.- Tests:
test/ai/reranker-readiness.test.ts,
test/rerank-no-key.serial.test.ts,test/hybrid-reranker-skipped.serial.test.ts,
test/doctor-reranker-health.test.ts,test/modes-report-reranker.test.ts,
test/init-reranker-default.test.ts; the sunset short-circuit suite now
pins the explicit-ZE scenario plus the live-default-after-the-date case.