Skip to content

BM25 k1 sweep + I(f) perturbation sensitivity #143

Description

@nikolay-e

Context

Two recorded-but-open sensitivity checks from v2: BM25 k1=2.5 sits above the classical range on a stated hypothesis (short identifier docs, late saturation) with no sweep; I(f) importance priors (0.10/0.15/1.0) are uncalibrated domain guesses.

Tasks

  • k1 sweep {1.2, 1.6, 2.0, 2.5, 3.0} x b {0.5, 0.75} on internal BM25 + RRF branch
  • I(f) perturbation +/-25% and +/-50% on impact-need scoring
  • Report deltas with CIs

Acceptance

  • both closed as tables in paper v3; defaults updated only on CI-excluding-zero improvements

Metadata

Metadata

Assignees

No one assigned

    Labels

    benchmarkpaperpost-v3Research/product backlog deferred past the v3 paper cycle

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions