Make backups more incremental friendly

[Kernald]

Integration name

backup

Link to integration documentation on our website

https://www.home-assistant.io/integrations/backup

Describe the enhancement

Currently, a back-up is composed of:

• A tar file, containing a JSON describing the backup, and another archive, containing:
• All the configuration
• The entire database
• Maybe more (e.g. media files people want to expose through the web UI), but that's beyond this conversation.

In my instance, my latest back-ups are 1.3GB - after working on making them smaller. My internet connection is not amazing, and uploading those previously 3.5GB archives every day was just unfeasible. Once extracted, I have ~1MB of configuration files, Zigbee database etc, and... 3.2GB of database, in a single blob.

Out of those 3.2GB, with the default recorder configuration of purging data older than 10 days, it seems fair to assume that ~90% is not changing day on day. But because it's in a single database file, and further compressed, it's virtually impossible to take advantage of any incremental back-up solution.

What I suggest is figuring out a way to split the data, so that the 90% of data that hasn't been changed doesn't change.

Use cases

Cloud back-ups would be much faster and cheaper. Assuming one wants to keep seven daily back-ups, two weekly back-ups and two monthly back-ups (that's what I typically keep for most of my services, at least), with the current implementation on my instance, it would take (7 + 2 + 2) * 3.5 = 38.5GB. With a more incremental-friendly approach, It would take:

• 3.5GB for the two monthly back-ups, as well as the first weekly back-up
• 7 days of changes between the two weekly back-ups, so the second weekly would take 3.5 / 10 * 7 = 2.45GB
• The oldest daily back-up is essentially the same as the most recent weekly back-up, hence doesn't really cost anything
• The remaining 6 daily back-ups take 3.5 / 10 * 6 = 2.1GB in total
  That's a total of 15.05GB - less than half of the storage used.

That's also an even more significant bandwidth saving, where in this scenario we'd only upload 3.5GB on the very first day and then ~10% of that every day (3.5 + 0.35 * 60 = 24.5GB), versus 3.5GB every day (213.5GB).

While those figures are obviously approximations, the trend is pretty obvious.

I understand that for a lot of people, uploading 3.5GB per day seems anecdotal, but not everybody is that lucky. Beyond that, as a cloud provider for Home Assistant back-ups, it would likely be a significant cost saving for you.

Anything else?

No response

[github-actions[bot]]

🏷️ I've automatically added the integration: backup label to help categorize this feature request.