Repository navigation
[Feature] add no compression option for backups #32383
matthewaveryusa
started this conversation in
Feature Request
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
I have searched the existing feature requests, both open and closed, to make sure this is not a duplicate request.
The feature
Even though backups are compressed with the gzip rsyncable option, my backups cannot efficiently dedup the backups. My setup is pretty simple: backups are saved to an s3 bucket using restic. In my tests, if backups were not compressed, I would be uploading ~20MB per day due to deduplication that restic does. With compression, I'm uploading 500MB per day. I'm not an expert in compression, but I think the factor at play here is the chunking that restic does is in segments too large compared to the gzip rsyncable chunks (or maybe just a different algorithm)
I'd like to proposed a fix, that I'll submit a PR for:
Add an option to not compress backups, with a toggle in the admin panel.
On the restore side, immich already accepts either .sql or .sql.gz.
Would you accept a PR for this option? I'll probably need to add the toggle in the mobile apps, but I'm not really an expert in that area.
Here's some evidence of the impact of the change: I changed from backing up compressed backups to adding a shim to decompress before uploading. the first upload was about twice the size, but then the incremental subsequent upload was tiny (12MB) compared to previous uploads.

Platform
All reactions