Skip to content

0.7.3 — the drop-in switch imported nothing, and score meant nothing across queries

Choose a tag to compare

@RamaAditya49 RamaAditya49 released this 08 Aug 14:23
· 20 commits to main since this release

Prompted by an external audit of b2d2fba. 61 of its claims were verified against HEAD before any of them was acted on: 32 held, 4 were already fixed, 18 held in part, and 7 were wrong. Everything below survived that check; the rejections are in PR #296 so nobody spends a day on them.

Fixed

  • Switching from @modelcontextprotocol/server-memory imported nothing, and said nothing. That server writes beside its own module when MEMORY_FILE_PATH is unset, so anyone who ran it the documented way has their graph in a hashed directory inside npm's _npx cache — never searched. An MCP server launched by a desktop client also inherits the client's working directory, so the one fallback was weakest exactly where it was relied on. Both populations imported zero entities in silence, which reads as Titen losing their memories. The search now covers the working directory, node_modules/@modelcontextprotocol/server-memory/dist beneath it, and every such install in the _npx cache, under both memory.jsonl and the older memory.json.
  • memory://knowledge-graph is served. resources/list returned -32601, so a client reading the graph as an MCP resource broke on the switch however well the nine tool names matched. initialize declares resources with subscribe: false: readable, no change notifications.
  • Every authenticated request paid a durable write to record that it had read. Measured on WAL with synchronous = FULL: fifty no-op CASE updates grow the WAL by 0 bytes, fifty real ones by 206,032 — one page and one fsync each, one billed write on D1. 25 of 27 GET routes paid it. Now bounded to one write per key per minute, still monotonic.
  • An imported credential colliding with another organization told the operator to retry forever — advice that can never succeed, because the collision is with another organization's row rather than a concurrent write.

Changed

  • score is comparable across queries (#227). Relevance was rescaled against the candidate set, pinning the best candidate to exactly 1 however poor the match, so rank 1 returned one distinct value across all 500 benchmark questions and threshold-based abstention was arithmetically impossible. It is now strength / (strength + 3.7) where strength is the bm25 magnitude per query term; rank 1 now returns 498 distinct values spanning 0.4875–0.6632. Pre-registered before any cell was scored: anchor recall@1 0.8800 → 0.8800, pooled 0.2460 → 0.2460, sign tests W0/L0/T500 at p = 1.0, compile p95 +1.21% / +0.67%.
  • That safety gate could not have failed, and the report says so. Both benchmark stores were ingested in one pass, so all five non-relevance components are constant across 89,467 packed items — with those constant the ordering is identical by arithmetic, making the unchanged recall a derivation rather than a finding. Where they vary the behaviour does change: at bm25 −60 against −45 the relevance gap compresses 0.250 → 0.018 and the order flips to the verified claim over the marginally better-matching asserted one. Intended, pinned by contract, and bounded — a genuinely weak match still loses to a strong one however trusted.
  • Two limits, stated rather than left to be discovered. 3.7 is calibrated on LongMemEval-S and BM25 is not portable across corpora, so the absolute band shifts with the corpus while the ordering does not. The vector arm's half of the change is unmeasured: both benchmark lanes ran with vectors disabled.

Removed

  • Five exports that shipped in the tarball with no caller anywhere in the source, tests, scripts or dashboard: ftsQuery, canReadRecord, recordEvent, param, FEEDBACK_ENDPOINT.

Upgrade notes

  • No API change and no migration. Anyone on 0.7.2 who switched from @modelcontextprotocol/server-memory and saw an empty store should restart titen mcp: the import runs on a store that has not yet imported, and it now looks where that server actually writes.
  • @types/node and @types/bun were never installed, so tsc could not resolve process, Bun, or any node:/bun: import — 791 reported errors, none of them bugs. src/ is now clean under tsc --noEmit with no any, @ts-ignore, or new cast added.

Full changelog · measurement · pre-registration