Skip to content

feat(cognition): singular/plural-fold knowledge-pack relevance scoring (recall quality) - #4250

Merged
rysweet merged 1 commit into
mainfrom
engineer/continuously-research-and-improve-your-own-cogn-70ab8541-1784267397-8ee1c9
Jul 17, 2026
Merged

feat(cognition): singular/plural-fold knowledge-pack relevance scoring (recall quality)#4250
rysweet merged 1 commit into
mainfrom
engineer/continuously-research-and-improve-your-own-cogn-70ab8541-1784267397-8ee1c9

Conversation

@rysweet

@rysweet rysweet commented Jul 17, 2026

Copy link
Copy Markdown
Owner

Summary

knowledge_context::relevance_score (consumed by enrich_planning_contextbase_type_turn on the reasoner's planning path) scored knowledge packs by exact whole-word overlap with the objective. A genuinely relevant pack was therefore missed whenever the objective and the pack name/description differed only by a regular English inflection — container vs containers, image vs images, library vs libraries, category vs categories. Such a pack scored below its true relevance (often zero), so it could be crowded out of the top-N (MAX_PACKS_PER_OBJECTIVE) cut and its knowledge never reached the planning prompt — deflating recall quality and thus reasoner reliability, two named axes of the standing cognition-improvement goal.

Relevance now matches an objective token against a pack word exactly or via a conservative lemma variant: the regular plural forms (+s/+es, and the singular obtained by stripping a trailing -s/-es) and the -y-ies pair. Folding is deliberately additive over — never a regression of — the whole-word rule adopted in #4241:

  • each generated variant must clear MIN_TOKEN_LEN, so a short token cannot fold onto a one-character fragment; and
  • a variant match only fires when the variant is actually a word in the pack, so non-plural words that merely end in s/es/y (class, focus, status, hierarchy) still match themselves and are never mangled onto an unrelated pack word.

Changed surfaces (criterion 2 — internal-only justification)

  • src/knowledge_context.rs — refines two PRIVATE helpers only: relevance_score and the new token_matches_pack. The public surface is unchanged: enrich_planning_context's signature and contract are intact (empty context on no match / propagated error on knowledge outage, capped at MAX_PACKS_PER_OBJECTIVE). No user-facing CLI/API/doc surface changes, so no user-docs update is required; the module rustdoc is updated in-place to document the folding policy and its precision guards.

Evidence

Criterion 1 — qa-team scenario

  • Added tests/qa-scenarios/knowledge-context-plural-folding-relevance.yaml (same outside-in pattern as the fix(cognition): whole-word, distinct-token knowledge-pack relevance scoring (recall quality) #4241 scenario). It drives the hermetic in-process knowledge_context unit tests and the consuming base_type_turn suite via cargo test, and rejects any non-zero exit. gadugi-test is not installed in this environment; the scenario's underlying cargo test commands are the real validation and are shown green below.

Criterion 3 — quality-audit (≥3 SEEK→VALIDATE→FIX cycles, clean final)

  • Cycle 1 (SEEK/VALIDATE/FIX): reviewed the folding for over-matching regressions vs the whole-word rule; validated class/focus/status are not mangled; FIX: collapsed nested if let blocks flagged by clippy into combinator form (strip_suffix(..).filter(..).is_some_and(..)).
  • Cycle 2 (SEEK/VALIDATE): checked the -y-ies and -s/-es precedence and the MIN_TOKEN_LEN stem guard; confirmed short tokens (by, is, os) cannot fold onto fragments; existing enrich_returns_empty_for_unrelated_objective (xyzzy plugh) still empty. No findings.
  • Cycle 3 (SEEK/VALIDATE, clean final): security/perf pass — pure string ops, bounded small-string allocation per token, off the recall hot path (planning enrichment, bounded pack list); non-test module lines 214 ≤ 250 spec budget. Zero critical/high; zero medium correctness/security findings.

Criterion 4 — CI green (local gates mirror CI)

  • cargo fmt --all -- --check — clean (pre-commit gate passed).
  • cargo clippy --release --no-deps -- -D warnings — clean (pre-commit gate passed).
  • cargo test --locked --lib knowledge_context13 passed; 0 failed (9 pre-existing + 4 new).
  • cargo test --locked --lib base_type_turn16 passed; 0 failed (consuming planning-context path unregressed).
  • pre-push push-stage gates — all passed.

New unit tests pinning the contract: relevance_score_folds_regular_plural_both_directions, relevance_score_folds_y_ies_variants, relevance_score_folding_does_not_mangle_s_ending_words, relevance_score_folding_preserves_whole_word_rule.

Criterion 6 — focused diff

  • 2 files: src/knowledge_context.rs (+ private helper + rustdoc + tests) and the new qa-scenario yaml. No unrelated edits.

Co-authored-by: Copilot 223556219+Copilot@users.noreply.github.com

…g (recall quality)

Planning-context knowledge-pack selection (`knowledge_context::relevance_score`,
consumed by `enrich_planning_context` → `base_type_turn`) scored packs by EXACT
whole-word overlap with the objective. A genuinely relevant pack was therefore
missed whenever the objective and the pack name/description differed only by a
regular English inflection — `container` vs `containers`, `image` vs `images`,
`library` vs `libraries`, `category` vs `categories`. Such a pack scored below
its true relevance (often zero), so it could be crowded out of the top-N
(`MAX_PACKS_PER_OBJECTIVE`) cut and its knowledge never reached the planning
prompt, deflating recall and thus reasoner reliability.

Relevance now matches an objective token against a pack word exactly OR via a
conservative lemma variant: the regular plural forms (`+s`/`+es`, and the
singular obtained by stripping a trailing `-s`/`-es`) and the `-y`↔`-ies` pair.
Folding is deliberately additive over — never a regression of — the whole-word
rule adopted in #4241:

- each generated variant must clear `MIN_TOKEN_LEN`, so a short token cannot
  fold onto a one-character fragment; and
- a variant match only fires when the variant is ACTUALLY a word in the pack,
  so non-plural words that merely end in `s`/`es`/`y` (`class`, `focus`,
  `status`, `hierarchy`) still match themselves and are never mangled onto an
  unrelated pack word.

Public surface unchanged (`enrich_planning_context` signature/contract intact);
this refines two PRIVATE helpers only (`relevance_score`, new
`token_matches_pack`). New unit tests pin the fold-both-directions, `-y`↔`-ies`,
no-mangle, and whole-word-preserved contract; a qa-scenario drives them
outside-in.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@github-actions

Copy link
Copy Markdown

📊 Coverage Summary

Generated by cargo llvm-cov --workspace --summary-only (nightly, excluding test files)

Module Lines Covered Coverage
Total 181976 151861 83.5%

Coverage data from CI run. Test files matching tests?/ are excluded from line counts.

@rysweet
rysweet merged commit 71fe605 into main Jul 17, 2026
16 checks passed
@rysweet
rysweet deleted the engineer/continuously-research-and-improve-your-own-cogn-70ab8541-1784267397-8ee1c9 branch July 17, 2026 07:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant