fix(cognition): dedup keywords + deterministic tie-break in knowledge-pack ranking (recall quality)

[rysweet]

Summary

Standing cognition goal — recall quality. Additive hardening of the knowledge-pack article ranking behind knowledge.query (native, Python-free reader in src/native_knowledge.rs), which feeds the reasoner's planning-context enrichment.

The keyword-coverage ranking (ORDER BY (title_hit*2 + content_hit*1) DESC) landed in #4281. This PR rebases onto that and closes two remaining recall-quality gaps it left:

1. Distinct keywords. Query keywords were not de-duplicated, so a word repeated in the question ("rust ... rust") double-counted in the coverage score and over-rewarded an article that merely mentions it once. query_pack_db now keeps only DISTINCT keywords, folded case-insensitively (to_ascii_lowercase) to match SQLite's ASCII-case-insensitive LIKE — making the "sum across distinct keywords" the ranking already documented actually hold.
2. Deterministic ordering. Equal-score matches had no tie-break, so their order fell back to SQLite's arbitrary rowid order (non-reproducible recall). The ORDER BY now ends , title ASC for run-to-run-stable ordering.

knowledge.query wire shape ({answer, sources, confidence}) is unchanged; only result ORDER improves.

Changed surfaces

• src/native_knowledge.rs — query_pack_db (case-insensitive keyword dedup), query_articles (deterministic title ASC tie-break + doc). Internal functions; RPC output shape unchanged.
• docs/reference/knowledge-pack-article-relevance-ranking.md (new) + mkdocs.yml nav entry — durable documentation of the full article-ranking behavior (coverage + title weighting + dedup + determinism).

Merge-ready evidence

1. Scenarios (tests) written, validated, run

gadugi-test is not provisioned in this engineer environment; the executable scenarios for this surface are the module's Rust unit tests. Added 2 additive tests alongside #4281's:

• query_articles_breaks_score_ties_by_title_ascending — three equal-score articles inserted reverse-alphabetically come back title ASC (proves determinism).
• query_pack_db_dedups_repeated_keywords — "rust rust ownership borrowing" ranks the higher-coverage article first; without dedup the repeated "rust" would have vaulted the single-keyword article to the top.

cargo test --release --lib -- native_knowledge
test result: ok. 20 passed; 0 failed; 0 ignored; 8790 filtered out

2. Docs

New durable reference doc docs/reference/knowledge-pack-article-relevance-ranking.md (doc_type: reference — not a point-in-time report, G4-compliant), wired into mkdocs.yml nav. tests/docs_integrity.rs: 4 passed; 0 failed.

3. Quality audit (SEEK → VALIDATE → FIX, ≥3 cycles, clean final)

• Cycle 1 — SEEK: a repeated query word double-counts in the coverage score. VALIDATE: query_pack_db_dedups_repeated_keywords (fails without the fix). FIX: case-insensitive keyword dedup.
• Cycle 2 — SEEK: equal-score matches order non-deterministically (flaky/irreproducible recall). VALIDATE: query_articles_breaks_score_ties_by_title_ascending. FIX: title ASC secondary sort.
• Cycle 3 — SEEK: does the dedup drop a genuinely distinct keyword or corrupt the answer message? VALIDATE: first-seen surface form + order preserved; existing query_pack_db_* and url-column tests still green (20/20). No regression.
• Final cycle (clean) — no new SQL-injection surface (keyword escaping unchanged); fallback nodes/entities tables gain no new failure mode (ORDER BY references only title/content, already required by the WHERE clause). cargo fmt --check clean; cargo clippy --release -- -D warnings and clippy --all-targets --all-features --locked -- -D warnings clean. Zero critical/high; zero medium correctness/security findings.

4. CI

Local gates green: cargo fmt --all -- --check; pre-commit clippy --release --no-deps -- -D warnings; pre-push race subset (cognitive_memory bootstrap memory_ipc memory_consolidation) 443 passed, 0 failed; clippy --all-targets --all-features --locked clean. Full CI runs on this PR.

6. Focused diff

3 files, +158/−4: src/native_knowledge.rs, docs/reference/knowledge-pack-article-relevance-ranking.md, mkdocs.yml. No unrelated edits. Rebased onto latest main (no conflict with #4281; layered strictly on top of it).

[github-actions]

📊 Coverage Summary

Generated by cargo llvm-cov --workspace --summary-only (nightly, excluding test files)

Module	Lines	Covered	Coverage
Total	184247	153985	83.6%

_{Coverage data from CI run. Test files matching tests?/ are excluded from line counts.}