-
Notifications
You must be signed in to change notification settings - Fork 24.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Better Incrementality for Snapshots of Unchanged Shards #52182
Better Incrementality for Snapshots of Unchanged Shards #52182
Conversation
…-source-only-snapshots
…-source-only-snapshots
@ywelsch @dnhatn Thanks for all the help here so far: I incorporated feedback and the work in #52694 into this PR now. Instead of dealing with all the details of the global- and local-checkpoints being equal etc. in the repository this is now all happening upstream in the => I think this one is good for another round of reviews whenever you have some time :) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I've left one ask for a test, o.w. looking good.
import static org.hamcrest.Matchers.is; | ||
|
||
@ESIntegTestCase.ClusterScope(scope = ESIntegTestCase.Scope.TEST, numDataNodes = 0) | ||
public class BlobStoreIncrementalityIT extends AbstractSnapshotIntegTestCase { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you also add a test that checks that force-merging is leading to another full snapshot?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sure thing, how about this one b508fb3
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
Thanks Yannick! |
Use sequence numbers and force merge UUID to determine whether a shard has changed or not instead before falling back to comparing files to get incremental snapshots on primary fail-over.
Use sequence numbers to determine whether a shard has changed or not instead before falling back to comparing files to get incremental snapshots on primary fail-over.