Unified Repository Design

[Lyndon-Li]

This PR add the initial design document for Unified Repository & Kopia Integration

[codecov-commenter]

Codecov Report

Merging #4926 (be820e0) into main (2464fcd) will decrease coverage by 0.01%.
The diff coverage is 50.00%.

@@            Coverage Diff             @@
##             main    #4926      +/-   ##
==========================================
- Coverage   41.29%   41.27%   -0.02%     
==========================================
  Files         211      211              
  Lines       18443    18445       +2     
==========================================
- Hits         7616     7614       -2     
- Misses      10254    10257       +3     
- Partials      573      574       +1     

Impacted Files	Coverage Δ
pkg/persistence/object_store.go	56.19% <50.00%> (-0.91%)	⬇️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 68730cb...be820e0. Read the comment docs.

[kaovilai]

Please add what to do when deleting backup containing snapshot that maybe a parent snapshot for another velero backup.
https://github.com/vmware-tanzu/velero/pull/4926/files#diff-2c3160d74871a65b38c165387821fb4627b1c6d2cd02a824ccfc0e376a4e5590

Since both restic and kopia are incremental by nature, subsequent snapshot only work if parent snapshot exists.

[reasonerjt]

I noticed an issue, that in the spec of the backup repository CR there is still resticIdentifier will that be renamed or how does kopia repo use this field?

[Lyndon-Li]

As our current discussion, the kopia path uses storageConfig (from BSL) instead of resticIdentifier/repoIdentifier, then the resticIdentifier/repoIdentifier are for Restic only. In this way, we could keep them as is since they are Restic specific.
On the other hand, if we want to make resticIdentifier/repoIdentifier generic for other purposes, we can change the name. For now, we don't have this requirement.

[Lyndon-Li]

@kaovilai Don't why I cannot reply your comments in place. Then let me make the answer here.

Both Kopia and Restic don't have the aforementioned snapshot chain problem, even though they support incremental snapshot/backup. The reason is, the incremental mechanism that is used by Kopia/Restic is file level and each snapshot contains a full view of its object data though the object data itself is shared across snapshots. Below are the details:

• Kopia/Restic checks the file's ctime, mtime, inode, etc. to decide whether a file has no change from its parent snapshots
• If the file is not changed, Kopia/Restic creates a new entry referring to the same object that is referred by the entry in the parent snapshot
• After that, there will be one new reference to the object
• When the parent snapshot is deleted, only the snapshot information is deleted. The deletion of the objects is the work of prune/maintenance
• When a prune/maintenance happens, since there is at least one reference to the object, the object is not deleted either

[kaovilai]

Thanks @Lyndon-Li