feat(corpus): refuse TF-IDF and BM25 as inferential weights

[seonghobae]

Summary

#37–#62 remain review-blocked (no independent APPROVE). #37 and #45 kept ready. Next no-0008 gap: inferential-weight refusal in existing corpus_split (does not recreate topic_measurement #48, concept_dictionary #53, or allocate 0008).

• refuse_inferential_retrieval_weight admits only group_normalized_ess and uniform
• TF-IDF and BM25 fail closed
• refuse_default_stopword_deletion refuses global stopword lists
• Computed RMSE of known membership shares is lower under allowed masses than under an L1-normalized TF-IDF surrogate

Test plan

• RED: inferential_weight_contract failed with E0432/E0599
• cargo test -p corpus_split --offline
• cargo clippy -p corpus_split --all-targets --offline -- -D warnings
• python3 scripts/validate_documentation.py and check_docstrings.py
• exact-head required Checks (do not treat queued evidence as passed)

Do not self-approve or merge. Prefer merge #37 then #45 when eligible. Do not allocate 0008.

[coderabbitai]

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 8db28420-1214-4ea1-b57c-3bad5c337231

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}