Add repair packs command to simplify recovery from damaged pack
#4530
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
7 tasks
MichaelEischer
force-pushed
the
repair-packs
branch
from
88ca464
to
a474df2
Compare
This allows recovering a repository from several cases of damaged blobs.
For now, the guide is only shown if the blob content does not match its hash. The main intended usage is to handle data corruption errors when using maximum compression in restic 0.16.0
MichaelEischer
force-pushed
the
repair-packs
branch
from
a474df2
to
d1d4510
Compare
MichaelEischer
commented
Oct 23, 2023
There was a problem hiding this comment.
LGTM.
I'll add a test later on. There's no documentation for now as the only intended usage is to repair the data corruption issues fixed by #4532, the check command provides the necessary instructions to fix those. A pack file can be damaged in other ways which have to be handled separately.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What does this PR change? What problem does it solve?
Adds a
repair packscommand that salvages all intact blobs from the specified pack files and updates the index accordingly. The salvaged pack files are written to the current folder, and the broken pack files are removed from the repository.Without this PR, the only way to recover from pack files with broken blobs is to either remove all snapshots that use those blobs followed by a prune run, removing the whole pack files or using a debug build of restic to run
restic debug examine --reupload-blobsfollowed by manually removing the pack file and repairing the index.The
repair packscommand is marked as experimental and introduces a stub feature flag implementation. The idea is to only enable experimental commands if the feature flag is set. The flag contains a version number which will be incremented in case of incompatible changes, this causes all invocations using the old flag version to fail. Feature flags and the corresponding commands may be removed/changed between restic versions without notice.Was the change previously discussed in an issue or on the forum?
No, however, #4523 might make it necessary to provide a more user friendly way to repair repositories.
Checklist
changelog/unreleased/that describes the changes for our users (see template).gofmton the code in all commits.