Skip to content

v2.0.0: stable BM25 term IDs, atomic writes, doctor

Choose a tag to compare

@ygtalp ygtalp released this 31 Jul 06:00
· 2 commits to main since this release

Requires a full reindex. After upgrading, run reindex(full=True) from a session. A plain reindex skips unchanged chunks and leaves the old sparse encoding in place — which is how a corrupt index survived the documented remedy.

Coming from 0.x, migrate the graph first with python -m rawthink_mcp.migrate --guess-domains --heal-dangling --apply. rawthink-doctor tells you what is still pending.

Why this release exists

BM25 term IDs were positions in a sorted vocabulary. One new token sorting early shifted every ID after it, so sparse vectors written before a reindex stopped matching queries encoded after it.

Nothing failed. RRF kept returning good results off the dense side, so search felt fine while half the retrieval system scored noise.

Measured on a real vault of 1,841 chunks:

before after
sparse MRR 0.1596 0.6259
sparse nDCG@10 0.1851 0.6320
sparse hit@10 5/19 14/19
dense MRR 0.6842 0.6842

Dense staying identical is what makes the sparse delta attributable to the fix.

A second finding

The measurement that confirmed the bug also surfaced something the audit had not predicted: 40 of 69 sessions in that vault had never been indexed. The files were on disk, search could never find them, and nothing said so. After indexing the sessions the evaluation needed, dense MRR went 0.6842 → 0.9474 and sparse 0.6259 → 0.8395.

The stale index was costing more than the encoding bug. Both were invisible, and together they read as one vague sense that search could be better. rawthink-doctor now checks for exactly this.

New

  • rawthink-doctor — nine checks with a fix line for each failure: vault, graph schema, BM25 state format, Qdrant, index coverage, vector dimension, Ollama, MCP registration.
  • Concurrent writes are safe. Graph mutations take a file lock and a thread lock. Measured before the fix, with three processes writing: one writer's twelve updates all vanished, silently.
  • Graceful shutdown releases the Qdrant directory lock and the graph file lock on SIGINT/SIGTERM. POSIX only — Windows terminates rather than signalling.
  • 104 tests, no Docker and no Ollama required, running on 3.10–3.13 in CI.
  • index_vault(force=True), so a full reindex actually re-encodes.

Changed

Term IDs are a hash of the token; format_version is 2 and legacy state is rejected rather than loaded. BM25 state lives with the vault instead of the package directory. Diagnostics go to stderr — they were going to stdout, which is the JSON-RPC channel, so the record of what an indexing run skipped never arrived anywhere. Graph search matches prefixes, not substrings. open_nodes is a pure read. get_session_chunks pages.

Reading

docs/postmortem-bm25-term-drift.md — how a defect stayed invisible for months, and what made it visible.

Full detail in CHANGELOG.md.