Skip to content

feat(models): make retrieval and fusion score semantics explicit (#90) - #142

Open
Muhammad-AbdullahGhani wants to merge 1 commit into
grayhatdevelopers:mainfrom
Muhammad-AbdullahGhani:feat/clear-ranking-scores
Open

feat(models): make retrieval and fusion score semantics explicit (#90)#142
Muhammad-AbdullahGhani wants to merge 1 commit into
grayhatdevelopers:mainfrom
Muhammad-AbdullahGhani:feat/clear-ranking-scores

Conversation

@Muhammad-AbdullahGhani

Copy link
Copy Markdown

Related issue

Closes #90

Summary

  • Add explicit ranking and score semantics across models, CLI, evidence delivery, and MCP tools:
    • SearchHit: added channel_rank, score_kind='ordering_only', score_direction='higher_is_better', score_conversion='negated_distance', distance_metric='cosine', and distance_direction='lower_is_better'.
    • FusedMoment: added combined_rank, score_kind='ordering_only', score_direction='higher_is_better', scoring_method='reciprocal_rank_fusion', contributing_channels, and channels_run.
    • FusionProvenance: added score_kind='ordering_only', score_direction='higher_is_better', and scoring_method='reciprocal_rank_fusion'.
    • EvidenceBoardCandidate: added score_kind='ordering_only' and score_direction='higher_is_better' when candidate score is present.
  • Backwards compatibility: all new fields include defaults and validators so existing callers and consumers remain fully compatible.
  • CLI: updated emit_search table caption and headers (Score (RRF ↑), Contributing Channels) to clarify that scores are ordering-only values rather than calibrated probabilities.
  • MCP: documented in search_moments and evidence_index that candidate scores and ranks are uncalibrated ordering values where higher is better.

Validation

  • pytest tests/test_search.py tests/test_search_fusion.py tests/test_query_service.py tests/test_evidence_delivery.py -p no:asyncio (37 passed)
  • pytest tests/test_cli.py -k search -p no:asyncio (2 passed)
  • ruff check src/vidxp/application_models.py src/vidxp/capabilities/search.py src/vidxp/search_fusion.py src/vidxp/cli_support.py src/vidxp/evidence_delivery.py src/vidxp/mcp.py tests/test_search.py tests/test_search_fusion.py (All checks passed)
  • ruff format --check src/vidxp/application_models.py src/vidxp/capabilities/search.py src/vidxp/search_fusion.py src/vidxp/cli_support.py src/vidxp/evidence_delivery.py src/vidxp/mcp.py tests/test_search.py tests/test_search_fusion.py (8 files already formatted)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Make retrieval and fusion scores clear to callers

2 participants