Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Snapshot/Restore: Add BWC layer to .si / segments_N hashing to identify segments accurately #7434

Closed
s1monw opened this issue Aug 25, 2014 · 0 comments · Fixed by #7436
Closed

Comments

@s1monw
Copy link
Contributor

s1monw commented Aug 25, 2014

This is a followup from #7351 where we added extra validation to .si and segments_n files. We now use their content as a has when we diff the snapshot against the store and wise versa. In order to not double the size of the snapshot when the hash is not present we should recalculate that "hash" from the blob store instead.

@s1monw s1monw self-assigned this Aug 25, 2014
s1monw added a commit to s1monw/elasticsearch that referenced this issue Aug 26, 2014
Due to additional safety added in elastic#7351 we compute now a strong hash for
.si and segments_N files which are compared during snapshot / restore.
Old snapshots don't have this hash which can cause unnecessary copying
of large amount of data. This commit adds the ability to fetch this
hash from the blob store if needed.

Closes elastic#7434
s1monw added a commit that referenced this issue Aug 26, 2014
Due to additional safety added in #7351 we compute now a strong hash for
.si and segments_N files which are compared during snapshot / restore.
Old snapshots don't have this hash which can cause unnecessary copying
of large amount of data. This commit adds the ability to fetch this
hash from the blob store if needed.

Closes #7434
s1monw added a commit that referenced this issue Sep 8, 2014
Due to additional safety added in #7351 we compute now a strong hash for
.si and segments_N files which are compared during snapshot / restore.
Old snapshots don't have this hash which can cause unnecessary copying
of large amount of data. This commit adds the ability to fetch this
hash from the blob store if needed.

Closes #7434
@clintongormley clintongormley changed the title [SNAPSHOT] add BWC layer to .si / segments_N hashing Snapshot/Restore: Add BWC layer to .si / segments_N hashing to identify segments accurately Sep 10, 2014
s1monw added a commit to s1monw/elasticsearch that referenced this issue Sep 24, 2014
Due to additional safety added in elastic#7351 we compute now a strong hash for
.si and segments_N files which are compared during snapshot / restore.
Old snapshots don't have this hash which can cause unnecessary copying
of large amount of data. This commit adds the ability to fetch this
hash from the blob store if needed.

Closes elastic#7434
@s1monw s1monw added the v1.3.3 label Sep 24, 2014
mute pushed a commit to mute/elasticsearch that referenced this issue Jul 29, 2015
Due to additional safety added in elastic#7351 we compute now a strong hash for
.si and segments_N files which are compared during snapshot / restore.
Old snapshots don't have this hash which can cause unnecessary copying
of large amount of data. This commit adds the ability to fetch this
hash from the blob store if needed.

Closes elastic#7434
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant