Improve index crash recovery #2394
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
In cases where the database only gets updated from partition transforms the index state file can go heavily out of sync. This becomes a problem in case of unexpected termination when the state can't get persisted, effectively resulting in data loss. This change adds calls to the persist function for that state in appropriate places so that the problematic situation is prevented moving forward.
Instead of the index.bin file, which might be out of sync
tobim
force-pushed
the
story/sc-35105/improve-index-crash-recovery
branch
from
b582750
to
cb6c011
Compare
We need to nudge them a bit so we can drop support for older partition versions more freely.
The condition for this was exactly inverted, so the command considered only oversized partitions.
dominiklohmann
approved these changes
Jul 4, 2022
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This causes VAST to restore from the available partition synopses on the filesystem instead of the partition synopses listed in the
index.binfile, which can easily go out of sync during crashes.This is a first change towards getting rid of the
index.binfile entirely.馃摑 Checklist
馃幆 Review Instructions
Review commit-by-commit. Test by terminating VAST ungracefully during a rebuild.