It'd be nice to save db backups occasionally

[netcreateorg]

Right now, we have occasional bugs that lead to corrupted data, and it is really hard to recover. Ideally, we'd have backups that we can go back to.

One option is to save every major change to a new DB, but presumably that'll get really big.

Can we check internally somehow if the data is good and then delete if it is? A record of old databases would be nice, but it's only really crucial because of the corruption / crash and we can't always predict those yet.

Flagging for @kalanicraig to weigh in

[benloh]

From email:
I think it should be relatively easy to implement. The question really is when you want the backup to be made. We can do it periodically, e.g. every minute, or on every save, which might be quite often (I can't remember off the top of my head, but we might be doing saves every second). I think the big worry would be how much space it can eat up over time, especially on DO instances. Granted, the files tend to be relatively small, but they can add up, especially if you're on a small drive.

[benloh]

@jdanish Can you please attach an example corrupted db file? (The "VF0i1.loki" file you had emailed to me was a weird binary with only 157 bytes...but maybe that's the resulting corrupted file? NOTE you'll probably need to zip it).

[netcreateorg]

VFOI1.loki.zip

[netcreateorg]

I'm thinking every update but will defer to Kalani.

[benloh]

One additional idea: We can save off backups to a backup folder to keep the runtime folder more pristine.

[benloh]

This is partially implemented: A backup is made whenever an import is completed.
The functionality for creating backups is there. The keys is figuring a mechanism for triggering that -- automatic? or manual?

Save this for the next round.

[netcreateorg]

Yes, definitely this. There’s an on-exit thing when someone leaves the browser and disconnects from the server, right? How often does that happen, and could that action trigger a backup? Or is that too late? (Like someone crashes after doing something stupid, triggers a backup and then the backup doesn’t get enough of what happened in the editing session?) —k

…

On Mar 14, 2022, at 11:41 AM, benloh ***@***.***> wrote: One additional idea: We can save off backups to a backup folder to keep the runtime folder more pristine. — Reply to this email directly, view it on GitHub <#213 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ALDNP5QYX6CQHJVBOE3YAQ3U75MZPANCNFSM5PCMCNQA>. Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>. You are receiving this because you authored the thread.