You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The read-only repository check is done: it hashes index/ and packs/ objects via Store.hash,
checks the index first and the packs only if the index is intact, and never rebuilds the index on
that path. But when it finds damage, there is no repair. Repository.check(repair=True) just logs
and returns:
# repository.py, check()elifrepair:
logger.error("Finished ... repository check, errors found (repository repair not implemented).")
and the corrupt-index branch has the placeholder for the missing part:
# TODO: --repair will rebuild the index from the packs here instead of stopping (refs #8572).logger.error("Repository index is corrupted and must be repaired; skipping the pack check.")
A repo with a corrupt index/ fragment or a corrupt pack therefore cannot be repaired at the
repository level. This ticket splits that work out of #9696 ("... to be continued").
Scope
Repository-level repair only: rebuilding the chunks index and salvaging packs. Archives repair
(dropping archives with corrupt/lost metadata, recreating lost entries) already exists, and missing
file-content chunks heal on the next borg create. Both are out of scope.
Work items, in order
Command surface. Decide before writing repair code: keep it under borg check --repair, or
split off a read-only borg check and a separate borg repair with explicit targets
(--index, --packs PACKID...) (borg2: improve borg check #9696). The items below assume the existing check paths; only
the CLI shape depends on this.
Index repair from headers. Rebuild the chunks index by range-reading pack object headers and
skipping the payloads, recording chunk id -> (pack id, offset, size). Reading full packs is too
slow to be the routine repair path (borg2: avoid doing an unnecessary rebuild of the index #8476, to pack or not to pack ... #8572). Replaces the TODO in the corrupt-index branch.
Do not trust the headers blindly (borg2: index rebuild trusts pack headers #9901). Headers are unauthenticated, so a tampered header
(wrong id or sizes) would poison the rebuilt index. Authenticate on first real read (AEAD binds
the id to the content) and/or verify ids for mismatching entries. Part of item 2, not a follow-up.
Pack repair. For a pack that fails Store.hash, keep every object that still
AEAD-authenticates in a new pack, drop the corrupt pack, and update the index. Objects that no
longer decrypt are already gone; drop them from the index so the affected archives report missing
chunks and heal on the next create (borg2: avoid doing an unnecessary rebuild of the index #8476, to pack or not to pack ... #8572). Needs item 2.
Keep the rebuilt index (borg2: check [--repair] improvements #8466).ArchiveChecker.finish() deletes the cached chunks index
after repair, so the next command pays for a slow rebuild. Write the index rebuilt in item 2 back
instead. Needs item 2.
borg2 check: implement repository repair (index and pack rebuild)
Refs #9696, #8466, #8476, #9901, #9825. Tracking: #9998.
Problem
The read-only repository check is done: it hashes
index/andpacks/objects viaStore.hash,checks the index first and the packs only if the index is intact, and never rebuilds the index on
that path. But when it finds damage, there is no repair.
Repository.check(repair=True)just logsand returns:
and the corrupt-index branch has the placeholder for the missing part:
A repo with a corrupt
index/fragment or a corrupt pack therefore cannot be repaired at therepository level. This ticket splits that work out of #9696 ("... to be continued").
Scope
Repository-level repair only: rebuilding the chunks index and salvaging packs. Archives repair
(dropping archives with corrupt/lost metadata, recreating lost entries) already exists, and missing
file-content chunks heal on the next
borg create. Both are out of scope.Work items, in order
Command surface. Decide before writing repair code: keep it under
borg check --repair, orsplit off a read-only
borg checkand a separateborg repairwith explicit targets(
--index,--packs PACKID...) (borg2: improve borg check #9696). The items below assume the existingcheckpaths; onlythe CLI shape depends on this.
Index repair from headers. Rebuild the chunks index by range-reading pack object headers and
skipping the payloads, recording
chunk id -> (pack id, offset, size). Reading full packs is tooslow to be the routine repair path (borg2: avoid doing an unnecessary rebuild of the index #8476, to pack or not to pack ... #8572). Replaces the TODO in the corrupt-index branch.
Do not trust the headers blindly (borg2: index rebuild trusts pack headers #9901). Headers are unauthenticated, so a tampered header
(wrong id or sizes) would poison the rebuilt index. Authenticate on first real read (AEAD binds
the id to the content) and/or verify ids for mismatching entries. Part of item 2, not a follow-up.
Pack repair. For a pack that fails
Store.hash, keep every object that stillAEAD-authenticates in a new pack, drop the corrupt pack, and update the index. Objects that no
longer decrypt are already gone; drop them from the index so the affected archives report missing
chunks and heal on the next create (borg2: avoid doing an unnecessary rebuild of the index #8476, to pack or not to pack ... #8572). Needs item 2.
Consume the persisted corrupt-pack list. A read-only check records corrupt pack ids in
cache/checked-packs(borg2 check: move from marker to set-of-checked-packs #9897; PR check: keep pack check results, add --max-age to reuse them #9925 makes them survive a completed check). Repair reads thatlist and touches only the flagged packs, then clears their records once fixed (borg2: improve borg check #9696). Needs
PR check: keep pack check results, add --max-age to reuse them #9925 plus items 2 and 4.
Keep the rebuilt index (borg2: check [--repair] improvements #8466).
ArchiveChecker.finish()deletes the cached chunks indexafter repair, so the next command pays for a slow rebuild. Write the index rebuilt in item 2 back
instead. Needs item 2.
Repair progress (Add progress indicator to repository check loop #9443, Progress of
Remote: checking segment file /whatever/data/17/17078...#8009). The read-only check shows per-loop percentages; the repairpaths (header scan, pack salvage, verify-data delete pass) need the same. Needs items 2 and 4.
Out of scope (tracked elsewhere)
borg2 checkreporting of missing chunks #9218,PR check: report missing chunks grouped as chunk -> files -> archives, #9218 #9965), Ctrl-C at safe boundaries (Ctrl-C does not interrupt borg check #7893, PR check: handle Ctrl-C at safe boundaries (#7893) #9966).
(borg2: improve dealing with repo corruption #9825).
Data-safety constraints
network or RAM errors must not destroy data.
borg creatededup (borg2: avoid doing an unnecessary rebuild of the index #8476).