Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use Lucene's query rescorer #6232

Closed
wants to merge 2 commits into from

Conversation

@s1monw
Copy link
Contributor

commented May 19, 2014

A query rescorer was added to lucene that basically does the same as our
rescorer. In-fact we designed it to work pretty similar. This commit
cuts over to the lucene version shipped with 4.8

@s1monw

This comment has been minimized.

Copy link
Contributor Author

commented May 19, 2014

I just integrated this and ran the tests... yet I ran into a lucene bug with the lucene rescorer... next time I will integrate earlier to make sure we catch those bugs earlier...

@mikemccand

This comment has been minimized.

Copy link
Contributor

commented May 19, 2014

Whoa, awesome! I didn't realize this would be so simple. Issue #5922 is open for this...

@nik9000

This comment has been minimized.

Copy link
Contributor

commented May 19, 2014

Nice!

s1monw added 2 commits May 19, 2014
Use Lucene's query rescorer
A query rescorer was added to lucene that basically does the same as our
rescorer. In-fact we designed it to work pretty similar. This commit
cuts over to the lucene version shipped with 4.8

Closes #5922
@s1monw

This comment has been minimized.

Copy link
Contributor Author

commented Jul 9, 2014

I just pushed a new commit for this. The bug that I referenced is fixed but there are several problems with the lucene APIs that I personally think it's not worth it unless lucene fixes it's APIs I had to do a lot of things on top that are really confusing IMO

@mikemccand

This comment has been minimized.

Copy link
Contributor

commented Jul 9, 2014

We can fix the Lucene APIs: they are experimental. E.g. we can change it to do the rescoring "in-place", but I find this sort of strange. The full-resort after rescoring only a subset is odd because you're comparing apples to oranges at that point...

Alternatively, we could just add a utility API in Lucene on top of its existing APIs to do the in-place rescoring that ES wants, i.e. extract the topN from the first pass, rescore all of those, put back in and do a full resort.

@clintongormley clintongormley added v1.5.0 and removed v1.4.0.Beta1 labels Sep 8, 2014

@s1monw s1monw assigned mikemccand and unassigned s1monw Sep 9, 2014

mikemccand added a commit that referenced this pull request Oct 18, 2014
Core: cutover to Lucene's query rescorer
This is functionally equivalent to before, so there should be no
user-visible impact, except I added a NOTE in the docs warning about
the interaction of pagination and rescoring.

Closes #6232

Closes #7707
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
4 participants
You can’t perform that action at this time.