MVP of encrypted backups resilient to bit rot (Data degradation / Data corruption)

[fititnt]

• https://en.wikipedia.org/wiki/Data_degradation
• https://en.wikipedia.org/wiki/Data_corruption
• https://pthree.org/2014/04/01/protect-against-bit-rot-with-parchive/
• https://www.reddit.com/r/DataHoarder/comments/8l0y7t/how_do_you_prevent_bit_rot_across_all_of_your/
• https://superuser.com/questions/1014071/how-to-protect-my-digital-data-from-solar-flares/1014179
  • https://en.wikipedia.org/wiki/Carrington_Event

---

TL;DR: Do shell scripting plus documentation of an minimum viable product (MVP) of backup archiving with bit rot / data degradation protection.

This is something I would do for other projects, but this one, since already likely to deal with encryption (and implicitly it means compression, and compression does not play nice with data recovery) it also means that the result is more likely to face some bit rot over the years (and some level of bit rot can make not only an small part of an backup like GBs of photos and documents, but the entire backup file!).

Not everything can have a live copy that could be rechecked all the time. . And by some histories on the web about bit rot, data on HDDs with 8+ years already can bit rot, so having the just single file replicated on 2-3 mediums may not in 10-15 years, but eventually all mediums could fail and the used format could not be recovered. Not good.

And even if just from time to time do a drill of copyng older files to newer drivers may not be good enough, (in special if they already compressed or encrypted), if someone else is not able to detect years sooner that one file already going bad. One scenario where even someone not able to open partially bit rotted file (like if a file is encrypted) but be able to recover before updating to a new storage medium seems totally win win. (And I say this because, as 2020, maybe even printing good quality books are more granted to last even centures than good quality digital storage mediums today are granted to persist just a few decades.

securebox-backup-*

I'm already drafting some POSIX shell scripts to allow remote backups on local machines (calling this as "securebox-backup-*"). Mostly testing on Debian-like distros but they are likely to work later on Fails. I think that for this specific issue it would mean doing some sort of parity at file level (something that could work even without RAID or a filesystem like ZFS).

100% POSIX shell scripting is harder than bash (and bash already is more complex for non-trivial script than any other scripting language. But since this is something I would expect to be able to work even 10-20 years later (maybe even have a copy of the entire operational system so in the worst case scenario even could allow someone to boot up from a virtual machine) the part about the scripting is not that bad.