Skip to content

Commit

Permalink
Skip identical entries when listing chunks
Browse files Browse the repository at this point in the history
The prune command can remove redundant chunks (chunks with the same chunk id
but at different subdirectory level).  However, if the same chunk appears
mutliple times in the listing returned by the storage, it will be treated as
a redundant chunk and thus removed.
  • Loading branch information
gilbertchen committed Sep 27, 2023
1 parent 4e9d2c4 commit d92b173
Showing 1 changed file with 6 additions and 0 deletions.
6 changes: 6 additions & 0 deletions src/duplicacy_snapshotmanager.go
Expand Up @@ -2427,7 +2427,13 @@ func (manager *SnapshotManager) pruneSnapshotsExhaustive(referencedFossils map[s
}

allFiles, _ := manager.ListAllFiles(manager.storage, chunkDir)
uniqueFiles := make(map[string]bool)
for _, file := range allFiles {
if _, found := uniqueFiles[file]; found {
continue
}
uniqueFiles[file] = true

if len(file) == 0 || file[len(file)-1] == '/' {
continue
}
Expand Down

2 comments on commit d92b173

@gilbertchen
Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This commit has been mentioned on Duplicacy Forum. There might be relevant details there:

https://forum.duplicacy.com/t/exhaustive-prune-on-idrive-e2-deletes-needed-chunks/7978/3

@gilbertchen
Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This commit has been mentioned on Duplicacy Forum. There might be relevant details there:

https://forum.duplicacy.com/t/duplicacy-cli-3-2-2-release/8019/1

Please sign in to comment.