[codex] Add RAG query debug details

[2002yy]

What changed

• Added RAG query debug diagnostics for retrieval mode, query terms, candidate count, returned count, per-result rank and score breakdown.
• Extended /rag/query and /rag responses with debug details.
• Added optional one-query evaluation when expected_sources are provided.
• Updated RAG docs and verified test counts.

Validation

• python -m pytest -q: 265 passed
• python -m ruff check .: passed
• detect-secrets scan over tracked source/doc/test/config paths: results empty
• python tools/package_project_helper.py . NUL 0: OK
• git diff --check: passed with CRLF warnings only
• python -m mypy --explicit-package-bases src: existing 18 soft-check errors remain

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ca59dd5644

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Report clipped hybrid vector scores

For hybrid searches, search_rag_index_hybrid() builds vector_scores from search_rag_index_vector(..., min_score=0.0), so negative cosine scores are omitted and contribute 0.0 to the combined score. This debug path instead records the raw cosine for every chunk, so any lexical hit whose vector similarity is negative will show a vector_score that was not actually used and the breakdown cannot explain the returned combined_score.

Useful? React with 👍 / 👎.