Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use dfs_query_then_fetch to make elasticsearch scoring more consistent #6112

Closed
diox opened this issue Oct 22, 2018 · 2 comments · Fixed by mozilla/addons-server#9932
Closed

Comments

@diox
Copy link
Member

diox commented Oct 22, 2018

By default, Elasticsearch scoring for documents is only calculated per shard for performance reasons. This means if you're unlucky and some of your shards contains more documents relevant to your query than the others, the score and therefore the position of each document in the results can be inconsistent with how relevant they are.

Elasticsearch provides an optional query type, dfs_query_then_fetch that makes the scoring consistent by using all shards to compute terms frequency and not just individual ones, at a small performance cost.

We briefly had that through a waffle but reverted it because the waffle itself had an additional performance cost, as it was executing a database/cache query in the search API. We should try enabling dfs_query_then_fetch unconditionally instead.

@diox
Copy link
Member Author

diox commented Nov 8, 2018

QA: this is hard to test, but make sure search is still behaving correctly and there are no huge unexpected performance regressions with the search API.

@AlexandraMoga
Copy link

I didn't find regressions around search relevancy or performance on AMO -dev.
I'm marking the issue as verified fixed on AMO -dev with FF63, WIn10x64.

@KevinMind KevinMind transferred this issue from mozilla/addons-server May 4, 2024
@KevinMind KevinMind added repository:addons-server Issue relating to addons-server migration:2024 labels May 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
No open projects
Development

Successfully merging a pull request may close this issue.

4 participants