fix(recall): cold pool selects by demonstrated reuse, not by self-rated importance - #37
Merged
Merged
Conversation
MXAntian
force-pushed
the
fix/cold-pool-selects-by-reuse
branch
from
September 1, 2026 13:24
e904426 to
cc0bccc
Compare
MXAntian
marked this pull request as ready for review
September 1, 2026 13:24
…ed importance The cold pool produces the "oh, I just remembered something" surface: when a recall comes back short, 25% of the time it adds 1-3 rows that are old, untouched, and still worth seeing. "Still worth seeing" was implemented as `importance >= 8`. Measured on a 9301-row library: that term excluded 441 of 1124 eligible rows, and the excluded set was led by the single most-recalled memory in the database — access_count=1879, importance=5. Also in there: acc=721, 634, 604, 529, 495, all rated 5-7 at write time. The gate was doing the opposite of its stated purpose. It filtered out the rows with the strongest evidence of being worth remembering, because someone typed a number when writing them and never revisited it. importance is 89% saturated at >=7 across the library, so as a selector it carries almost no signal — and where it does discriminate, it discriminates wrong. The decay floor already does this job on earned signal. decay_score is w(age) * reuseBoost(access_count), so a row cold for 30 days only still holds decay_score >= 0.3 if it was reused heavily. Measured on the same library, every row reaching the pool had access_count >= 8, average 135. Selecting by reuse is what "still important" was always trying to approximate — we just had a proxy sitting right there and used a self-report instead. Dropping the term widens the pool 683 -> 1124 and admits exactly the rows with the best track record. ## Index The index definition lived in three places that all had to agree: schema.sql (fresh DBs), migrations/003 (canonical), and an inline copy in index.mjs (old-DB upgrades). All three led on importance and were partial on `importance >= 8`, so with that term gone from the query SQLite could no longer use any of them and would have fallen back to a scan. All three now lead on last_accessed, which is what the pool range-filters. The index.mjs migration block re-runs on every initMemory(), so the DROP is guarded on the old definition actually being present — an unguarded drop would rebuild the index at every process start, at a cost that grows with the library. Caught by reading the surrounding block rather than by a test; then verified both directions: fresh DB: run 1 -> 0 rebuilds, run 2 -> 0 (all three definitions agree) old DB: run 1 -> 1 rebuild, run 2 -> 0 (upgrades once, then never) ## Also - split the query out as selectColdPoolCandidates so it is testable without fighting Math.random() and ORDER BY RANDOM() ## Verification Red-then-green checked properly rather than assumed. With `importance >= 8` restored, exactly the one assertion that matters fails (5 pass / 1 fail) — the other five stay green, so they are not accidentally coupled to it. Full suite green: cold-pool 6 · recall-contract 14 · ranking-importance 5 · anchor-pinned 4 · level-migration 10 · supersede-shrink 24 · recall-endpoint 17 · query-rewrite 6 · memory-health 68 · locations 51 · provenance 25 · encoding-damage 18 · injection-hygiene 14 · hooks 12 · series-detection 11. [prediction] 修复: 冷池复现从"半个库里随机捞"变成"复用记录最强的那批里随机捞", 长期没碰但真正常用的条目重新有机会浮上来 [prediction] 风险: 池子从 683 涨到 1124,单条被抽中的概率下降约 40%。若实际感受 变成"总冒出不相干的",说明 decay_score>=0.3 这个地板对本库太松,调地板而不是 把 importance 加回来 [prediction] 验证: 下次 recall 返回 surfaced_random 时,命中条目的 access_count 应普遍 >= 8;若出现 access_count 很低的条目,说明 decay 公式或地板需要复核 Co-Authored-By: 千夏 <qianxia@clawgamers.com>
MXAntian
force-pushed
the
fix/cold-pool-selects-by-reuse
branch
from
September 1, 2026 13:29
cc0bccc to
74e039f
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The cold pool produces the "oh, I just remembered something" surface: when a recall comes back short, 25% of the time it adds 1–3 rows that are old, untouched, and still worth seeing.
"Still worth seeing" was implemented as
importance >= 8.What that gate actually did
Measured on a 9301-row library: the term excluded 441 of 1124 eligible rows — and the excluded set was led by the single most-recalled memory in the database.
The gate was doing the opposite of its stated purpose: it filtered out the rows with the strongest evidence of being worth remembering, because someone typed a number when writing them and never revisited it.
importanceis 89% saturated at>= 7across the library, so as a selector it carries almost no signal — and where it does discriminate, it discriminates wrong.What replaces it
Nothing — the decay floor already does this job, on earned signal.
decay_scoreisw(age) * reuseBoost(access_count), so a row cold for 30 days only still holdsdecay_score >= 0.3if it was reused heavily. Measured on the same library, every row reaching the pool hadaccess_count >= 8, average 135.Selecting by reuse is what "still important" was always trying to approximate. The proxy was sitting right there; the code used a self-report instead.
Pool: 683 → 1124, gaining exactly the rows with the best track record.
Also
selectColdPoolCandidatesso it's testable without fightingMath.random()andORDER BY RANDOM()idx_mem_surface_pool. Its old definition led onimportanceand was partial onimportance >= 8; with that term gone from the query, SQLite could no longer use it and would have fallen back to a scan. Now leads onlast_accessed, which is what the query range-filters.Verification
Red-then-green checked properly rather than assumed. With
importance >= 8restored, exactly the one assertion that matters fails — the other five stay green, so they aren't accidentally coupled to it:Full suite after the change — all green: cold-pool 6 · recall-contract 14 · ranking-importance 5 · anchor-pinned 4 · level-migration 10 · supersede-shrink 24 · recall-endpoint 17 · query-rewrite 6 · hooks 12.
Risk
The pool grows ~65%, so any single row's odds of being drawn fall about 40%. If the felt experience becomes "it keeps surfacing irrelevant things", that means the
decay_score >= 0.3floor is too loose for this library — tune the floor, don't putimportanceback.Stacks conceptually with #36 (same theme: signals that measure the author's intent rather than the system's behaviour). Branched off
mainindependently; CI hunks don't overlap.