Skip to content

Tweak solr weights/boosts - switch from additive boosting to multiplactive boosting#12357

Open
cdrini wants to merge 1 commit intomasterfrom
11777/feature/tweak-solr-weights
Open

Tweak solr weights/boosts - switch from additive boosting to multiplactive boosting#12357
cdrini wants to merge 1 commit intomasterfrom
11777/feature/tweak-solr-weights

Conversation

@cdrini
Copy link
Copy Markdown
Collaborator

@cdrini cdrini commented Apr 13, 2026

Closes #11777

Technical

  • Previously, we used the bf parameter for our boosting, but this applied additive boosting, meaning the boosting could outweight the relevance quite drastically. Switching to boost gives us multiplicative boosting, which is recommended. See Additive Boosts vs Multiplicative Boosts
  • Note there is some performance concern since we are (1) using a lot of fields in our qf, where previously we only had a few, and (2) using more pf fields. If there is too much of a performance cost, we can switch back to using text ; there is still a big improvement from the boosting changes alone.

Testing

Improves upon all classes in our evaluation spreadsheet. C in the following tables (Note A is the same as C, but using text instead of the explicit listing of fields.)

image

You can also compare results before/after here: https://codepen.io/cdrini/full/wvJqzaK

Screenshot

Stakeholders

Copilot AI review requested due to automatic review settings April 13, 2026 16:28
@cdrini cdrini added State: Blocked Work has stopped, waiting for something (Info, Dependent fix, etc. See comments). [managed] Needs: Testing labels Apr 13, 2026
@github-actions github-actions bot added the Priority: 2 Important, as time permits. [managed] label Apr 13, 2026
@cdrini cdrini force-pushed the 11777/feature/tweak-solr-weights branch from fa46472 to 54c1707 Compare April 13, 2026 16:34
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adjusts Work Search’s Solr eDismax tuning to use multiplicative boosting (via boost) instead of additive boosting (via bf) to reduce over-weighting of popularity signals relative to textual relevance.

Changes:

  • Replaces bf-based additive boosts with an eDismax boost function expression.
  • Expands/adjusts qf and phrase-boost parameters (pf, pf2) to change match weighting and proximity scoring.

Comment thread openlibrary/plugins/worksearch/schemes/works.py Outdated
Comment thread openlibrary/plugins/worksearch/schemes/works.py Outdated
Comment thread openlibrary/plugins/worksearch/schemes/works.py Outdated
@cdrini cdrini force-pushed the 11777/feature/tweak-solr-weights branch from 3594574 to ddf5888 Compare April 13, 2026 16:40
@cdrini
Copy link
Copy Markdown
Collaborator Author

cdrini commented Apr 13, 2026

Patching this out now for a bit to monitor performance.

@cdrini cdrini added the Patch Deployed This PR has been deployed to production independently, outside of the regular deploy cycle. label Apr 13, 2026
@mekarpeles mekarpeles self-assigned this Apr 13, 2026
@cdrini cdrini removed the Patch Deployed This PR has been deployed to production independently, outside of the regular deploy cycle. label Apr 14, 2026
@mekarpeles
Copy link
Copy Markdown
Member

Here's an example @cdrini showed where the search was "The secret of secrets" which didn't appear within the first 15 results on (before) and is 3rd on (after)

image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Needs: Testing Priority: 2 Important, as time permits. [managed] State: Blocked Work has stopped, waiting for something (Info, Dependent fix, etc. See comments). [managed]

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Re-tune work search ranking parameters

3 participants