Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

check -files downloads the same chunk multiple times #477

Open
crabmanX opened this issue Aug 5, 2018 · 0 comments

Comments

Projects
None yet
1 participant
@crabmanX
Copy link

commented Aug 5, 2018

when running duplicacy check on a backup with many small files (e.g. /etc), duplicacy downloads the same chunks multiple times:

man_db.conf
Fetching chunk b6532942e52733aa2f268ae5804d347f80c8e72bd7e52a88811d6e652db85efa
Chunk b6532942e52733aa2f268ae5804d347f80c8e72bd7e52a88811d6e652db85efa has been downloaded
mdadm.conf
Fetching chunk b6532942e52733aa2f268ae5804d347f80c8e72bd7e52a88811d6e652db85efa
Chunk b6532942e52733aa2f268ae5804d347f80c8e72bd7e52a88811d6e652db85efa has been downloaded
mime.types
Fetching chunk b6532942e52733aa2f268ae5804d347f80c8e72bd7e52a88811d6e652db85efa
Chunk b6532942e52733aa2f268ae5804d347f80c8e72bd7e52a88811d6e652db85efa has been downloaded
mke2fs.conf
Fetching chunk b6532942e52733aa2f268ae5804d347f80c8e72bd7e52a88811d6e652db85efa
Chunk b6532942e52733aa2f268ae5804d347f80c8e72bd7e52a88811d6e652db85efa has been downloaded
mkinitcpio.conf
Fetching chunk b6532942e52733aa2f268ae5804d347f80c8e72bd7e52a88811d6e652db85efa
Chunk b6532942e52733aa2f268ae5804d347f80c8e72bd7e52a88811d6e652db85efa has been downloaded
mkinitcpio.conf.pacnew
Fetching chunk b6532942e52733aa2f268ae5804d347f80c8e72bd7e52a88811d6e652db85efa
Chunk b6532942e52733aa2f268ae5804d347f80c8e72bd7e52a88811d6e652db85efa has been downloaded
mkisolinux.conf
Fetching chunk b6532942e52733aa2f268ae5804d347f80c8e72bd7e52a88811d6e652db85efa
Chunk b6532942e52733aa2f268ae5804d347f80c8e72bd7e52a88811d6e652db85efa has been downloaded

So in my case, checking a 116MB backup of 5409 small files takes about 20 hours, while just downloading the entire backup is a matter of seconds.

For starters, it would be nice if duplicacy could cache the chunks or ideally check all files contained in a chunk at once.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.