fix(cognition): whole-word, marker-safe episodic keyword recall (recall quality) - #4265
Merged
rysweet merged 1 commit intoJul 17, 2026
Conversation
…all quality)
`LibraryCognitiveMemory::search_episodes_by_keywords` matched every keyword by
raw case-insensitive substring, so a short clean token embedded in the
interior/suffix of an unrelated episode word ("test" in "latest", "decision"
in "indecision") spuriously recalled off-topic episodes for the
natural-language callers (creative_ideas: "meeting"/"conversation"/"decision").
The sibling ranked path already uses a word-boundary gate; the flat scan was
left on substring only because its bracketed-marker callers
(memory_consolidation::reflection_lessons) depend on exact-substring match.
Partition the query keywords by shape: a clean alphanumeric keyword is matched
at a word boundary via the existing `shares_word_prefix` gate (inflection-
tolerant, so plural/verb recall is preserved), while a phrase or bracketed
provenance marker keeps the exact substring semantics reflection_lessons dedup
and recurring-failure counting rely on. A `has_clean` guard skips the
word-boundary tokenization on the marker-only path (count_recurring_failures
scans with limit = u32::MAX).
Reuses main's tokenize_words/shares_word_prefix helpers (no second policy).
Adds a live-backend integration test module and a qa-team scenario; updates the
adapter architecture doc and the trait doc.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
rysweet
force-pushed
the
engineer/continuously-research-and-improve-your-own-cogn-70ab8541-1784289647-70b3cd
branch
from
July 17, 2026 13:32
a8fac1d to
30b2eee
Compare
📊 Coverage Summary
Coverage data from CI run. Test files matching |
rysweet
deleted the
engineer/continuously-research-and-improve-your-own-cogn-70ab8541-1784289647-70b3cd
branch
July 17, 2026 15:15
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Improves episodic recall quality on the flat keyword scan
LibraryCognitiveMemory::search_episodes_by_keywords. It matched every keyword byraw case-insensitive substring (
content.contains(kw)), so a short clean tokenembedded in the interior/suffix of an unrelated episode word spuriously recalled
off-topic episodes for its natural-language callers — e.g.
decision⊂ "indecision",test⊂ "latest",own⊂ "download". This dilutes keyword recall and,downstream, reasoner reliability.
The sibling ranked path (
recall_episodes_ranked) already uses a word-boundarygate (
shares_word_prefix/tokenize_words). The flat scan was deliberately left onsubstring only because its bracketed-marker callers
(
memory_consolidation::reflection_lessons) depend on exact-substring match. This PRcloses that gap safely: it partitions query keywords by shape so clean tokens get
the word-boundary gate while markers keep substring — resolving the exact constraint
the substring path was preserved for, and reusing main's existing helpers (no second
matching policy).
The fix (marker-safe, reuses the established word-boundary gate)
creative_ideas("meeting"/"conversation"/"decision") and
tokenize_objectiveemit): matched at aword boundary via the existing
shares_word_prefix(a query token must be aprefix of a whole content word). Inflection-tolerant, so
meetingstill recalls"meetings" and
deploystill recalls "deployed" (no Fix the episodic-recall-returns-zero defect in Simard's cognitive memory. SYMPTOM: Memory preparation logs "preparation: 5 procedures, 0 episodes recalled (0 raw, 0 session-filtered)" and "prepared c #2299 "0 raw" regression). Killsthe interior/suffix false positives.
[reflect-occ=…],[reflect-key=…|…]): keeps legacy case-insensitive substring semantics; thereflection_lessonsdedup and recurring-failure counting are byte-for-byte unchanged.has_cleanflag skips the word-boundary tokenization entirely on themarker-only path (
count_recurring_failuresscans withlimit = u32::MAX).episode.
Merge-ready evidence
1. qa-team scenario (written, validated, run)
tests/qa-scenarios/whole-word-episode-recall.yamlgadugi-test validate -f tests/qa-scenarios/whole-word-episode-recall.yaml --strict→
✓ Scenario "whole-word-episode-recall" is valid(1 valid, 0 invalid)gadugi-test run -d tests/qa-scenarios -s whole-word-episode-recall→
✓ Passed: 1 ✗ Failed: 0(drives the live-backend contract, the pureword_boundary_gate_tests, and the fullmemory_consolidationsuite proving theexact-marker
reflection_lessonsdedup is unregressed).2. Docs updated (user-facing surfaces)
docs/architecture/cognitive-memory-library-adapter.md: updated thesearch_episodes_by_keywordsbehavior rows and added a "Keyword matching:word-boundary vs substring" subsection documenting the clean-token vs marker split.
src/cognitive_memory/mod.rs: trait doc forsearch_episodes_by_keywordsupdated todescribe the word-boundary + marker-safe semantics.
Changed surfaces:
search_episodes_by_keywordsrecall behavior for clean keywords(now word-boundary). Marker/phrase callers are behavior-preserving (substring path
unchanged).
3. quality-audit (≥3 SEEK→VALIDATE→FIX cycles, ended clean)
partition keywords into clean (word-boundary) vs raw (substring).
non-alphanumeric chars → routed to the substring path; confirmed by the full
memory_consolidationsuite (148 pass, incl.reflection_lessonsdedup and the PR-Crecall/count e2e tests).
u32::MAXscan (needlesstokenization) → FIX:
has_cleanguard skipsshares_word_prefixwhen no cleankeyword is present.
tokenize_words/shares_word_prefix(avoid a second, divergent matching policy); re-ran targeted suites,
clippy --lib --tests -- -D warnings(clean),fmt --check(clean). Zero critical/high; zeromedium correctness/security findings.
Local validation (all green):
cognitive_memory::(129),memory_consolidation(148),creative_ideas(74),memory_ipc::(68), the newcognitive_memory::tests_whole_word_episode_recall(6),docs_integrity(4).Pre-push gate: race-subset suite 443 passed, 0 failed and
clippy --all-targets --all-features --locked -- -D warnings(clean). Pre-commit gate:release
clippy --release --no-deps -- -D warnings(clean).4. CI
full clippy).
main(resolved a conflict from the concurrently-mergedrecall_episodes_rankedword-boundary work); full CI monitored on this PR and will beconfirmed 100% green before requesting review/merge.
6. Focused diff
5 files, no unrelated edits:
src/cognitive_memory/library_adapter.rs— partition + reuse ofshares_word_prefixsrc/cognitive_memory/mod.rs— trait doc + test module registrationsrc/cognitive_memory/tests_whole_word_episode_recall.rs— new integration test moduletests/qa-scenarios/whole-word-episode-recall.yaml— new qa scenariodocs/architecture/cognitive-memory-library-adapter.md— behavior/divergence docStanding-goal alignment
A durable code improvement to recall quality (cognition), extending the just-merged
word-boundary gate to the last substring recall path in a marker-safe way — consistent
with the perpetual "research and improve your own cognition" goal (durable PRs, not
snapshot docs).