Skip to content

Alternative approach to LUCENE-8962 - #1576

Closed
s1monw wants to merge 1 commit into
apache:masterfrom
s1monw:LUCENE-8962
Closed

Alternative approach to LUCENE-8962#1576
s1monw wants to merge 1 commit into
apache:masterfrom
s1monw:LUCENE-8962

Conversation

@s1monw

@s1monw s1monw commented Jun 13, 2020

Copy link
Copy Markdown
Member

this is just a prototype - only intended for discussion

@s1monw s1monw changed the title lternative approach to LUCENE-8962 Alternative approach to LUCENE-8962 Jun 13, 2020
@s1monw

s1monw commented Jun 13, 2020

Copy link
Copy Markdown
Member Author

see #1552 for reference

@s1monw
s1monw marked this pull request as draft June 13, 2020 11:19

boolean await(long timeout, TimeUnit unit) {
for (OneMerge merge : merges) {
if (merge.await(timeout, unit) == false) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks suspicious when there is more than one merge. Shouldn't the timeout decrease as time is used up by earlier merges?

In practice, when is there more than one? I've been confused on this matter when I developed a custom MP/MS.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

in a real change that's correct. in a prototype as this is it's really just there to visualize the idea. I didn't do this on purpose to not discuss impl details. that's not the point of this it's really just a PR to make commenting simpler.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I fixed this in the followup PR #1585

@msfroh

msfroh commented Jun 16, 2020

Copy link
Copy Markdown
Contributor

This approach makes sense to me.

I like how much simpler the addition of MergeSpecification.await() makes things, versus the CountDownLatch hackery of the previous approach. Also, updatePendingMerges returning the MergeSpecification is much cleaner than explicitly computing a merge from within prepareCommitInternal.

@s1monw

s1monw commented Jun 16, 2020

Copy link
Copy Markdown
Member Author

@msfroh I opened #1585 to make it easier to do this.

@s1monw

s1monw commented Jun 17, 2020

Copy link
Copy Markdown
Member Author

superseded by #1585

@s1monw s1monw closed this Jun 17, 2020
} else {
spec = mergePolicy.findMerges(trigger, segmentInfos, this);
switch (trigger) {
case COMMIT:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is an inconsistency here that suggests something is wrong, or at least confusing enough to deserve a comment. For case COMMIT, we call a findFullFlushMerges. Shouldn't it be on case FULL_FLUSH to be consistent with the method we are calling? Or should findFullFlushMerges be called findCommitMerges?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Full flush happens for refresh and commit.

But we have only implemented "merge on commit" so far.

I would love to also add "merge on refresh", but until we do so, I think it's correct for IndexWriter to separate out the COMMIT case here like this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants