fix(rag): decouple hybrid recall from top_k for stable merged ranking…

[thomascolden585-svg]

Summary

Fixes mixed (hybrid) RAG retrieval so changing only Top K no longer changes which segments participate in score fusion, which could reorder the head and make the first N results differ between a small Top K and a large Top K for the same query.

Problem

Hybrid search runs vector and full-text in parallel, deduplicates, then merges scores (weighted score or reranker). The final top_k was also used as the per-channel limit for each sub-retriever. A segment can sit below k in both channels but still get a high combined score. With a small k it never entered the candidate pool; with a larger k it did—so the merged top results were inconsistent with user expectations (reported in #35482).

Solution

• Introduce a dedicated per-channel recall for hybrid: min(200, max(50, final_top_k)).
• Use it for embedding_search and full_text_index_search in hybrid mode.
• Keep post-merge cut at the user’s top_k via DataPostProcessor.invoke(..., top_n=top_k).

Trade-offs

Slightly more work per hybrid query (larger per-channel fetch) in exchange for stable, correct fusion behavior.

Test plan

• uv run pytest tests/unit_tests/core/rag/datasource/test_datasource_retrieval.py::test_hybrid_recall_top_k_for_merge_contract -q
• uv run pytest tests/unit_tests/core/rag/retrieval/test_dataset_retrieval.py -k hybrid -q
• Manual: same query, hybrid + weighted / rerank, Top K 3 vs 8 — first 3 should match (same ordering and segments).

Related

• Closes / fixes RAG mixed retrieval problem #35482