feat(cognition): singular/plural-fold knowledge-pack relevance scoring (recall quality) - #4250
Merged
rysweet merged 1 commit intoJul 17, 2026
Conversation
…g (recall quality) Planning-context knowledge-pack selection (`knowledge_context::relevance_score`, consumed by `enrich_planning_context` → `base_type_turn`) scored packs by EXACT whole-word overlap with the objective. A genuinely relevant pack was therefore missed whenever the objective and the pack name/description differed only by a regular English inflection — `container` vs `containers`, `image` vs `images`, `library` vs `libraries`, `category` vs `categories`. Such a pack scored below its true relevance (often zero), so it could be crowded out of the top-N (`MAX_PACKS_PER_OBJECTIVE`) cut and its knowledge never reached the planning prompt, deflating recall and thus reasoner reliability. Relevance now matches an objective token against a pack word exactly OR via a conservative lemma variant: the regular plural forms (`+s`/`+es`, and the singular obtained by stripping a trailing `-s`/`-es`) and the `-y`↔`-ies` pair. Folding is deliberately additive over — never a regression of — the whole-word rule adopted in #4241: - each generated variant must clear `MIN_TOKEN_LEN`, so a short token cannot fold onto a one-character fragment; and - a variant match only fires when the variant is ACTUALLY a word in the pack, so non-plural words that merely end in `s`/`es`/`y` (`class`, `focus`, `status`, `hierarchy`) still match themselves and are never mangled onto an unrelated pack word. Public surface unchanged (`enrich_planning_context` signature/contract intact); this refines two PRIVATE helpers only (`relevance_score`, new `token_matches_pack`). New unit tests pin the fold-both-directions, `-y`↔`-ies`, no-mangle, and whole-word-preserved contract; a qa-scenario drives them outside-in. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
📊 Coverage Summary
Coverage data from CI run. Test files matching |
rysweet
deleted the
engineer/continuously-research-and-improve-your-own-cogn-70ab8541-1784267397-8ee1c9
branch
July 17, 2026 07:18
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
knowledge_context::relevance_score(consumed byenrich_planning_context→base_type_turnon the reasoner's planning path) scored knowledge packs by exact whole-word overlap with the objective. A genuinely relevant pack was therefore missed whenever the objective and the pack name/description differed only by a regular English inflection —containervscontainers,imagevsimages,libraryvslibraries,categoryvscategories. Such a pack scored below its true relevance (often zero), so it could be crowded out of the top-N (MAX_PACKS_PER_OBJECTIVE) cut and its knowledge never reached the planning prompt — deflating recall quality and thus reasoner reliability, two named axes of the standing cognition-improvement goal.Relevance now matches an objective token against a pack word exactly or via a conservative lemma variant: the regular plural forms (
+s/+es, and the singular obtained by stripping a trailing-s/-es) and the-y↔-iespair. Folding is deliberately additive over — never a regression of — the whole-word rule adopted in #4241:MIN_TOKEN_LEN, so a short token cannot fold onto a one-character fragment; ands/es/y(class,focus,status,hierarchy) still match themselves and are never mangled onto an unrelated pack word.Changed surfaces (criterion 2 — internal-only justification)
src/knowledge_context.rs— refines two PRIVATE helpers only:relevance_scoreand the newtoken_matches_pack. The public surface is unchanged:enrich_planning_context's signature and contract are intact (empty context on no match / propagated error on knowledge outage, capped atMAX_PACKS_PER_OBJECTIVE). No user-facing CLI/API/doc surface changes, so no user-docs update is required; the module rustdoc is updated in-place to document the folding policy and its precision guards.Evidence
Criterion 1 — qa-team scenario
tests/qa-scenarios/knowledge-context-plural-folding-relevance.yaml(same outside-in pattern as the fix(cognition): whole-word, distinct-token knowledge-pack relevance scoring (recall quality) #4241 scenario). It drives the hermetic in-processknowledge_contextunit tests and the consumingbase_type_turnsuite viacargo test, and rejects any non-zero exit.gadugi-testis not installed in this environment; the scenario's underlyingcargo testcommands are the real validation and are shown green below.Criterion 3 — quality-audit (≥3 SEEK→VALIDATE→FIX cycles, clean final)
class/focus/statusare not mangled; FIX: collapsed nestedif letblocks flagged byclippyinto combinator form (strip_suffix(..).filter(..).is_some_and(..)).-y↔-iesand-s/-esprecedence and theMIN_TOKEN_LENstem guard; confirmed short tokens (by,is,os) cannot fold onto fragments; existingenrich_returns_empty_for_unrelated_objective(xyzzy plugh) still empty. No findings.Criterion 4 — CI green (local gates mirror CI)
cargo fmt --all -- --check— clean (pre-commit gate passed).cargo clippy --release --no-deps -- -D warnings— clean (pre-commit gate passed).cargo test --locked --lib knowledge_context— 13 passed; 0 failed (9 pre-existing + 4 new).cargo test --locked --lib base_type_turn— 16 passed; 0 failed (consuming planning-context path unregressed).New unit tests pinning the contract:
relevance_score_folds_regular_plural_both_directions,relevance_score_folds_y_ies_variants,relevance_score_folding_does_not_mangle_s_ending_words,relevance_score_folding_preserves_whole_word_rule.Criterion 6 — focused diff
src/knowledge_context.rs(+ private helper + rustdoc + tests) and the new qa-scenario yaml. No unrelated edits.Co-authored-by: Copilot 223556219+Copilot@users.noreply.github.com