Skip to content

fix(rerank): normalize NVIDIA logit scores - #2496

Merged
lyingbug merged 1 commit into
Tencent:mainfrom
BigFishDreamWater:codex/fix-nvidia-rerank-logits
Aug 3, 2026
Merged

fix(rerank): normalize NVIDIA logit scores#2496
lyingbug merged 1 commit into
Tencent:mainfrom
BigFishDreamWater:codex/fix-nvidia-rerank-logits

Conversation

@BigFishDreamWater

@BigFishDreamWater BigFishDreamWater commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

中文

修复内容

  • 将 NVIDIA reranker 响应中的原始 logit 映射为 [0, 1] 概率型 RelevanceScore
  • 使用数值稳定的 sigmoid 实现,避免极大正负 logit 的指数溢出。
  • 新增真实 HTTP 适配器路径的回归测试,覆盖正、零、负 logit,校验范围和相对排序。

复现说明

  1. 配置 NVIDIA rerank provider,并使接口返回多个高 logit,例如 23.05.02.0
  2. 修复前,适配器将这些原始值直接写入 RelevanceScore。聊天链把它们按概率型分数合成并截断到 1,多个候选会出现相同的最终相关性,MMR 丢失模型的排序信息。
  3. 修复后,适配器在边界处执行 sigmoid 映射。23.00.0-23.0 分别转换为接近 10.50 的值,仍保留严格的相对顺序。
    NVIDIA 响应里的字段明确叫 logit,但代码直接映射到 RelevanceScore:[nvidia_reranker.go (line 42)]
    随后不做转换地复制给公共结果:[nvidia_reranker.go (line 126)
    聊天 pipeline 再把它代入:
    composite := 0.6modelScore + 0.3baseScore + 0.1*sourceWeight
    然后强制限制到 [0,1]:[rerank.go (line 439)。
    假设原始召回分数 baseScore=0.5、普通知识库来源权重为 1:
    logit = 23 → 0.6×23 + 0.3×0.5 + 0.1 = 14.05 → 截断为 1
    logit = 5 → 0.6×5 + 0.3×0.5 + 0.1 = 3.25 → 截断为 1
    logit = 2 → 0.6×2 + 0.3×0.5 + 0.1 = 1.45 → 截断为 1
    logit = -23 → -13.55 → 截断为 0
    原来 23 > 5 > 2,进入 MMR 后却全是 1。

验证

  • go test ./internal/models/rerank ./internal/application/service/chat_pipeline -count=1
  • go vet ./internal/models/rerank

未修改仓库文档。

English

Fix

  • Map the raw NVIDIA reranker logit response field to a probability-like [0, 1] RelevanceScore.
  • Use a numerically stable sigmoid implementation to avoid exponential overflow for extreme logits.
  • Add an adapter-level regression test through a real HTTP path, covering positive, zero, and negative logits, score range, and order preservation.

Reproduction

  1. Configure the NVIDIA rerank provider and return several high logit values, for example 23.0, 5.0, and 2.0.
  2. Before this change, the adapter copied those raw values into RelevanceScore. The chat pipeline then combined them as probability-like scores and capped them at 1, giving multiple candidates the same final relevance and causing MMR to lose the model ranking signal.
  3. With this change, the adapter applies sigmoid at the provider boundary. 23.0, 0.0, and -23.0 become values near 1, 0.5, and 0, respectively, while preserving strict relative order.

Verification

  • go test ./internal/models/rerank ./internal/application/service/chat_pipeline -count=1
  • go vet ./internal/models/rerank

No repository documentation was changed.

Checklist

  • git diff --check upstream/main...HEAD
  • Changed Go files are formatted
  • Targeted tests for affected packages pass
  • Self-reviewed the change
  • Added regression coverage

@BigFishDreamWater BigFishDreamWater changed the title fix(rerank): normalize NVIDIA logit scores / 归一化 NVIDIA logit 分数 fix(rerank): normalize NVIDIA logit scores Aug 3, 2026
@lyingbug
lyingbug merged commit 31a2358 into Tencent:main Aug 3, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants