Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Share TermsEnum if possible while applying DV updates [LUCENE-8602] #9648

Closed
asfimport opened this issue Dec 11, 2018 · 5 comments
Closed

Share TermsEnum if possible while applying DV updates [LUCENE-8602] #9648

asfimport opened this issue Dec 11, 2018 · 5 comments

Comments

@asfimport
Copy link

Today we pull a new terms enum when we apply DV updates even though the
field stays the same which is the common case. Benchmarking this on a
larger term dictionary with a significant number of updates shows a
2x improvement in performance.


Migrated from LUCENE-8602 by Simon Willnauer (@s1monw), resolved Dec 11 2018
Pull requests: apache/lucene-solr#523

@asfimport
Copy link
Author

ASF subversion and git services (migrated from JIRA)

Commit d08e2d4 in lucene-solr's branch refs/heads/master from @s1monw
https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=d08e2d4

LUCENE-8602: Share TermsEnum if possible while applying DV updates

Today we pull a new terms enum when we apply DV updates even though the
field stays the same which is the common case. Benchmarking this on a
larger term dictionary with a significant number of updates shows a
2x improvement in performance.

@asfimport
Copy link
Author

ASF subversion and git services (migrated from JIRA)

Commit 01ad36d in lucene-solr's branch refs/heads/branch_7x from @s1monw
https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=01ad36d

LUCENE-8602: Share TermsEnum if possible while applying DV updates

Today we pull a new terms enum when we apply DV updates even though the
field stays the same which is the common case. Benchmarking this on a
larger term dictionary with a significant number of updates shows a
2x improvement in performance.

@asfimport
Copy link
Author

ASF subversion and git services (migrated from JIRA)

Commit d08e2d4 in lucene-solr's branch refs/heads/jira/http2 from @s1monw
https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=d08e2d4

LUCENE-8602: Share TermsEnum if possible while applying DV updates

Today we pull a new terms enum when we apply DV updates even though the
field stays the same which is the common case. Benchmarking this on a
larger term dictionary with a significant number of updates shows a
2x improvement in performance.

@asfimport
Copy link
Author

asfimport commented Dec 13, 2018

ASF subversion and git services (migrated from JIRA)

Commit 4272c63 in lucene-solr's branch refs/heads/master from @s1monw
https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=4272c63

#9654: Extract utility class to iterate over terms docs

Today we re-implement the same algorithm in various places
when we want to consume all docs for a set/list of terms. This
caused serious slowdowns for instance in the case of applying
updates fixed in LUCENE-8602. This change extracts the common
usage and shares the interation code including logic to reuse
Terms and PostingsEnum instances as much as possble and adds
tests for it.

@asfimport
Copy link
Author

asfimport commented Dec 13, 2018

ASF subversion and git services (migrated from JIRA)

Commit 8f4e4a4 in lucene-solr's branch refs/heads/branch_7x from @s1monw
https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=8f4e4a4

#9654: Extract utility class to iterate over terms docs

Today we re-implement the same algorithm in various places
when we want to consume all docs for a set/list of terms. This
caused serious slowdowns for instance in the case of applying
updates fixed in LUCENE-8602. This change extracts the common
usage and shares the interation code including logic to reuse
Terms and PostingsEnum instances as much as possble and adds
tests for it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant