Skip to content

fix(cognition): whole-word merged in episode classifier (recall quality / distillation fact-yield) - #4245

Merged
rysweet merged 1 commit into
mainfrom
engineer/continuously-research-and-improve-your-own-cogn-70ab8541-1784255034-58f073
Jul 17, 2026
Merged

fix(cognition): whole-word merged in episode classifier (recall quality / distillation fact-yield)#4245
rysweet merged 1 commit into
mainfrom
engineer/continuously-research-and-improve-your-own-cogn-70ab8541-1784255034-58f073

Conversation

@rysweet

@rysweet rysweet commented Jul 17, 2026

Copy link
Copy Markdown
Owner

Summary

Perpetual-cognition improvement on the recall-quality / distillation-fact-yield axis.

The episode-ingestion classifier (memory_consolidation::classifier::classify) decides — at every store_episode intake site — whether an event is operational noise (DROP), low-value bookkeeping (DOWN-SCOPE), or a meaningful episodic worth full-importance storage (STORE). Distillation later mines the STORED, high-importance episodes into semantic facts, so a misclassification here propagates straight into recall top-k.

The defect

Rule 3's durable-completion signal scored a single token with a raw substring content_lc.contains("merged"). That fired inside the git-status vocabulary unmerged / submerged — which name the opposite of a completed merge (an outstanding, un-completed one) — promoting benign/negative bookkeeping to a durable ActionCompleted episode at 0.7 importance. Distillation would then mine that phantom completion into a phantom "action completed" fact, polluting recall top-k and dragging recall precision down.

The fix

merged now matches at word boundaries via a new contains_word helper, aligning this seam with the word-boundary policy already adopted by the same classifier's failure-signal pass (#4080) and the knowledge-pack relevance scorer (#4241). unmerged/submerged down-scope to the operational tier; a genuine whole-word merged still classifies as a durable completion, so no real completion changes disposition. The multi-word phrases (opened pr, pull request) cannot embed in a single word, so they intentionally remain substring checks.

Merge-ready evidence

Criterion 1 — qa-team scenarios (validated + run):

  • New outside-in scenario tests/gadugi/episode-classifier-whole-word-completion.yaml drives the contract over the real classifier (headline asserts unmerged_paths_are_not_a_durable_completion ... ok, whole_word_merged_is_still_a_durable_completion ... ok, and the unchanged meaningful_completed_action_is_stored, plus test result: ok + exit 0).
  • The gadugi-test binary is not installed in this engineering environment, so gadugi-test validate/run could not be executed here. The scenario was validated structurally (YAML parses; schema-parity with the sibling distill-fact-yield-informative-content.yaml) and its underlying command cargo test --locked --lib memory_consolidation::classifier passes (20 passed; 0 failed).

Criterion 2 — docs / changed-surface justification: No user-facing surface changed. classify signature/contract is unchanged; this refines a private helper and one internal branch only. No CLI/operator/README surface touched → internal-only, no doc update required.

Criterion 3 — quality-audit (SEEK→VALIDATE→FIX): 3 cycles. Cycle 1: found the unmerged/submerged substring false-positive → added contains_word + tests. Cycle 2: verified no other single-token durable-completion signal has the same class (multi-word phrases are embed-safe; handoff has no embedding words and would regress handoffs under exact-match, so left as substring; recipe only routes within the failure branch). Cycle 3: caught + fixed a doc-comment inaccuracy (importance band 0.9→0.7). Final cycle clean: zero critical/high; zero medium correctness/security findings.

Criterion 4 — CI green (local gates mirror CI):

  • cargo fmt --all -- --check ✅ (pre-commit + pre-push)
  • cargo clippy --release --no-deps -- -D warnings ✅ (pre-commit)
  • cargo clippy --all-targets --all-features --locked -- -D warnings ✅ (pre-push)
  • Race-subset release tests cognitive_memory bootstrap memory_ipc memory_consolidation ✅ (pre-push)
  • cargo test --locked --lib memory_consolidation → 148 passed; 0 failed.

Criterion 6 — focused diff: 3 files (classifier.rs production, classifier_tests.rs tests, one new gadugi scenario). No unrelated edits.

Tests added

  • unmerged_paths_are_not_a_durable_completionunmerged/submerged do NOT store as ActionCompleted; down-scope to the operational tier.
  • whole_word_merged_is_still_a_durable_completion — a genuine whole-word merged still classifies as ActionCompleted (no regression of the real signal).

Standing perpetual goal — durable code improvement, not a snapshot doc.

Co-authored-by: Copilot 223556219+Copilot@users.noreply.github.com

…lity / fact-yield)

The episode-ingestion classifier (`memory_consolidation::classifier::classify`)
decides, at every `store_episode` intake site, whether an event is operational
noise (DROP), low-value bookkeeping (DOWN-SCOPE), or a meaningful episodic worth
full-importance storage (STORE). Distillation later mines the STORED,
high-importance episodes into semantic facts, so a misclassification here
propagates straight into recall top-k.

Rule 3's durable-completion signal scored a single token with a raw-substring
`content_lc.contains("merged")`. That fired inside the git-status vocabulary
`unmerged` / `submerged` — which name the OPPOSITE of a completed merge (an
outstanding, un-completed one) — promoting benign/negative bookkeeping to a
durable `ActionCompleted` episode at 0.7 importance. Distillation would then
mine that phantom completion into a phantom "action completed" fact, polluting
recall top-k and dragging recall precision down.

The `merged` signal now matches at WORD BOUNDARIES via a new `contains_word`
helper, aligning this seam with the word-boundary policy already adopted by the
same classifier's failure-signal pass (#4080) and the knowledge-pack relevance
scorer (#4241). `unmerged`/`submerged` down-scope to the operational tier; a
genuine whole-word `merged` still classifies as a durable completion, so no real
completion changes disposition. The multi-word phrases (`opened pr`,
`pull request`) cannot embed in a single word, so they remain substring checks.

Internal surface only — `classify` signature/contract unchanged, no CLI/operator
surface touched. New unit tests pin both the negative (`unmerged`/`submerged`)
and positive (whole-word `merged`) contract; a qa-team gadugi scenario drives
them outside-in.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@github-actions

Copy link
Copy Markdown

📊 Coverage Summary

Generated by cargo llvm-cov --workspace --summary-only (nightly, excluding test files)

Module Lines Covered Coverage
Total 184497 153960 83.4%

Coverage data from CI run. Test files matching tests?/ are excluded from line counts.

@rysweet
rysweet merged commit ef07442 into main Jul 17, 2026
16 checks passed
@rysweet
rysweet deleted the engineer/continuously-research-and-improve-your-own-cogn-70ab8541-1784255034-58f073 branch July 17, 2026 03:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant