Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

LUCENE-9324: Add an ID to SegmentCommitInfo #1434

Merged
merged 17 commits into from Apr 18, 2020
Merged

Conversation

s1monw
Copy link
Member

@s1monw s1monw commented Apr 16, 2020

We already have IDs in SegmentInfo, as well as on SegmentInfos which are useful to uniquely identify segments and entire commits. Having IDs on SegmentCommitInfo is be useful too in
order to compare commits for equality and make snapshots incremental on generational files.
This change adds a unique ID to SegmentCommitInfo starting from Lucene 8.6. Older segments won't have an ID until the segment receives an update or a delete even if they have been opened and / or committed by Lucene 8.6 or above.

@s1monw s1monw requested a review from jpountz April 16, 2020 13:11
@s1monw
Copy link
Member Author

s1monw commented Apr 16, 2020

Note: there is still a NOCOMMIT in this pr regarding BWC. It's just an idea we can build on or even move to using null as an indicator that there is no id.

@s1monw s1monw requested a review from jpountz April 16, 2020 20:49
@s1monw
Copy link
Member Author

s1monw commented Apr 17, 2020

@mikemccand @jpountz I pushed new commits that preserves the null value as much as necessary until the segment actually changes. This way we will be able to fall back to different methods to figure out if segments are the same. please take another look

@s1monw s1monw requested a review from jpountz April 17, 2020 17:58
Copy link
Member

@mikemccand mikemccand left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @s1monw! Getter looks much more like a normal getter now :)

s1monw added a commit that referenced this pull request Apr 18, 2020
We already have IDs in SegmentInfo, as well as on SegmentInfos which are useful
to uniquely identify segments and entire commits. Having IDs on
SegmentCommitInfo is be useful too in order to compare commits for equality and
make snapshots incremental on generational files.  This change adds a unique ID
to SegmentCommitInfo starting from Lucene 8.6. Older segments won't have an ID
until the segment receives an update or a delete even if they have been opened
and / or committed by Lucene 8.6 or above.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
3 participants