fix(cognition): drop stopwords in knowledge-pack relevance scoring (recall quality)

[rysweet]

Summary

Standing cognition-improvement goal — recall quality axis.

Planning-context knowledge-pack selection (knowledge_context::relevance_score, consumed by enrich_planning_context → base_type_turn) scored packs by whole-word, distinct-token overlap between the objective and each pack's name + description (post #4241), but it never dropped English function words. Because this seam admits 2-char tokens (MIN_TOKEN_LEN == 2), short function words (of, to, in, on, at, by, is, it) plus longer ones (the, and, for, with, …) that coincidentally appeared in both the objective and an off-topic pack's text each added +1 to that pack's relevance. Those filler hits could crowd a genuinely relevant pack out of the MAX_PACKS_PER_OBJECTIVE cut, injecting off-topic knowledge into the planning prompt and degrading reasoner reliability.

relevance_score now excludes a curated closed-class function-word stopword set before scoring, aligning this seam with the stopword policy already adopted for episodic/procedural recall by memory_consolidation::tokenize_objective (which never needed the 2-char entries because it drops <3-char tokens first). The set is deliberately conservative: only grammatical function words, never open-class task terms (fix/bug/test) and never legitimate short technical topics (go, os, io, ml, js, db, ci, cd), so a real match is never dropped.

Public surface unchanged — enrich_planning_context signature/contract intact; this refines a private helper only.

Changed surfaces

• src/knowledge_context.rs — private relevance_score + new private RELEVANCE_STOPWORDS const / is_relevance_stopword helper. No public/user-facing surface changed, so no user docs require updates (internal-only justification per criterion 2). The behavior change is documented in-module and pinned by tests.
• tests/qa-scenarios/knowledge-context-stopword-relevance.yaml — new outside-in scenario.

Evidence

1. qa-team scenario

Added tests/qa-scenarios/knowledge-context-stopword-relevance.yaml, mirroring the known-good knowledge-context-whole-word-relevance.yaml (#4241) format. It drives the hermetic in-process contract tests and a regression pass over the consuming turn path:

• cargo test --locked --lib knowledge_context → test result: ok. 12 passed; 0 failed
• cargo test --locked --lib base_type_turn → test result: ok. 16 passed; 0 failed

(Note: gadugi-test is not installed on this engineering host; the scenario is a thin wrapper whose underlying cargo test commands are the real assertions and were run directly — both green. YAML validated as well-formed.)

2. Docs

No user-facing surface changed (private helper only). In-module rustdoc updated to document the stopword policy and its alignment with memory_consolidation::tokenize_objective.

3. quality-audit (>=3 SEEK→VALIDATE→FIX cycles, clean final)

• Cycle 1 — SEEK: stopword list over-reach. VALIDATE: list restricted to standard closed-class function words; legit short tech tokens (go/os/io/ml/js/db/ci/cd) explicitly excluded and pinned by relevance_score_keeps_short_technical_tokens. No code fix.
• Cycle 2 — SEEK: filter correctness/placement. VALIDATE: objective tokens lowercased before stopword membership check; distinct-token (HashSet) property preserved. No fix.
• Cycle 3 — SEEK: does pack side need filtering? VALIDATE: filtering objective tokens alone is sufficient and minimal (a pack stopword can only match an objective stopword, which is already removed). Readability nit (rustfmt collapsed group comments inline) → FIX: removed redundant inline comments.
• Final cycle: cargo fmt --check clean, cargo clippy --lib --no-deps -- -D warnings clean, tests green. Zero critical/high; zero medium correctness/security findings.

4. CI green (local mirror)

• cargo fmt --all -- --check ✓ (pre-commit + pre-push)
• cargo clippy --release --no-deps -- -D warnings ✓ (pre-commit)
• cargo clippy --all-targets --all-features --locked -- -D warnings ✓ (pre-push)
• pre-push race-subset release tests (cognitive_memory bootstrap memory_ipc memory_consolidation) ✓
• Full lib suite: cargo test --locked --lib → 8664 passed; 0 failed; 7 ignored

6. Focused diff

Two files, +177/−5. No unrelated edits.

Co-authored-by: Copilot 223556219+Copilot@users.noreply.github.com