Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Address Database-Corruption in the script and also add a disclaimer in the corresponding yt-video #15

Closed
madejackson opened this issue Sep 13, 2023 · 4 comments

Comments

@madejackson
Copy link

madejackson commented Sep 13, 2023

Thanks for this awesome Script. Important question:

Did you and how did you solve the issue with database-corruption?

When doing a snapshot of a live database (fe. sqlite3 for radarr, sonarr etc), the snapshop holds a version of a possibly corrupt database and hence is possibly useless: https://www.google.com/search?q=sqlite+zfs+snapshot

@madejackson
Copy link
Author

madejackson commented Sep 19, 2023

Ok, So to answer my own question: Database corruption isn't addressed and it'll happen eventually. Somebody run into this the hard way on reddit. I won't post the link as I am against using reddit.

I think SpaceinvaderOne should update his video to address this specific issue. Otherwise newbies won't have any recoverable backups.

@madejackson madejackson changed the title Is Database-Corruption in this script avoided? Address Database-Corruption in the script and also add a disclaimer in the video? Sep 19, 2023
@madejackson madejackson changed the title Address Database-Corruption in the script and also add a disclaimer in the video? Address Database-Corruption in the script and also add a disclaimer in the corresponding yt-video Sep 19, 2023
@Octopusprime83
Copy link

Octopusprime83 commented Sep 29, 2023

Awesome script, and I have a solution for this that perhaps you could incorporate into the script, so its a couple things, First create a variable that when set to yes would call a "does docker container DB?" function. Add this as part of the pre-check, the code would be docker stop $(docker ps -a -q -f "label=zfs_snap_stop") This will check all dockers for the label of "zfs_snap_stop" (no quotes) and if present will stop the container. Then after snaps have been created add a docker start $(docker ps -a -q -f "label=zfs_snap_stop") to the script post snap competition. At the moment I'm using this as 2 separate scripts that run on either side of my appdata autosnap that uses your script and it works, the only issue is I have to guess how long your script will run and provide time for it to complete before restarting the containters.

In this way we can start/ stop containers with DB's with this script by simply applying labels to them (feel free the change the label to whatever you want). This should allow the pending changes to be written to the DB on container shutdown to avoid the DB and changes in ram from getting out of sync due to snapshotting. This issue is especially bad when nextcloud is working with a MS SQL or MYSQL DB backend.

@SpaceinvaderOne
Copy link
Owner

Yes thats a great idea to add a label to various containers. I will implement that for sure.
For nextcloud etc i am a big fan of using a dedicated database backup solution so database doesnt need to be shutdown
or example db-backup which you can find on CA This can backup multiple DB bervers (Influx, Maria/Mysql/Mongo/Postgres/Redis/Rethink)

@Jarikk
Copy link

Jarikk commented Oct 3, 2023

This issue is especially bad when nextcloud is working with a MS SQL or MYSQL DB backend.

Hi. At least for MySQL/MariaDB database containers I've added a new functionality to the script - please check my PR #21.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants