auto backup #1077

mstormi · 2020-08-05T22:26:33Z

Signed-Off-By: Markus Storm markus.storm@gmx.net

holgerfriedrich · 2020-08-06T05:12:20Z

@mstormi I am in doubt that copying a mounted (writable) partition of a running system on block basis (dd) could work. You will not get a snapshot, but likely in inconsistent state of the filesystem (it might still look ok and by mountable, but the content of the files might point to something completely different due to modifications of the fs during copy).

My recommendation would be to try an rsync (file basis) with some special handling of system directories plus some extra efforts to get it bootable etc.

mstormi · 2020-08-06T08:43:39Z

Thanks for the warning. Yes I know all those horror stories as I've been doing backups for >20 years in different organizations.
But I'm sure it works fine. I've successfully dd'ed /dev/mmcblk0 a couple of times in the past on my production system and never had issues.
First, it's done during low-traffic times, and most remaining traffic will be on ZRAM'ed directories (the raw device which is what I dump is not changed with ZRAM active).

Also, if you look at the code (beware, that's rapidly prototyped only so not ready yet), you'll see that I prepared for a combination of dd and rsync. My idea is to do raw dumps just once a month and rsync just the diffs all the other days so even if there was a problem with dd, it should get fixed on a subsequent rsync run.

mstormi · 2020-08-06T08:44:37Z

BTW, supposed to handle #1044

ecdye · 2020-08-07T01:48:50Z

While making this be sure to keep #625 in mind as it will soon be the standard across all the code once #1055 is merged.

ecdye · 2020-08-12T01:03:25Z

We should create a size chart for storage space recommendations for the backup device. with descriptions of how much space will be used for each piece of the backup solution.

ecdye · 2020-08-12T01:06:51Z

Also, why not use Amanda to create the RAW disk backups?

mstormi · 2020-08-12T07:24:09Z

We should create a size chart for storage space recommendations for the backup device

Don't think so. Required storage amount is unknown as it depends on compression, runtime config and more.
I've added to the docs (and code) that the mirror SD must be at least twice as large as the internal one.
The amount isn't important as SDs are only available in sizes of powers of 2 anyway.

Also, why not use Amanda to create the RAW disk backups?

Thought about that but that would require additional storage space on the order of (at least) the internal SD thus raise HW requirements

* implement ext card reader selection based on lsblk Signed-off-by: Markus Storm <markus.storm@gmx.net> * add mirrordrive boot parameter Signed-off-by: Markus Storm <markus.storm@gmx.net> * shellchecked Signed-off-by: Markus Storm <markus.storm@gmx.net> * add mount unit for /backup Signed-off-by: Markus Storm <markus.storm@gmx.net> * add mount unit for /backup Signed-off-by: Markus Storm <markus.storm@gmx.net> * add mirror setup during unattended install Signed-off-by: Markus Storm <markus.storm@gmx.net> * shellchecked Signed-off-by: Markus Storm <markus.storm@gmx.net> * add zramsync.service Signed-off-by: Markus Storm <markus.storm@gmx.net> * add zramsync.service Signed-off-by: Markus Storm <markus.storm@gmx.net> * first version mirror_SD Signed-off-by: Markus Storm <markus.storm@gmx.net> * first version mirror_SD [skip ci] Signed-off-by: Markus Storm <markus.storm@gmx.net> * first version mirror_SD [skip ci] Signed-off-by: Markus Storm <markus.storm@gmx.net> * first version mirror_SD [skip ci] Signed-off-by: Markus Storm <markus.storm@gmx.net> * first version mirror_SD [skip ci] Signed-off-by: Markus Storm <markus.storm@gmx.net> * define space for use by Amanda [skip ci] Signed-off-by: Markus Storm <markus.storm@gmx.net> * mount and rsync in /usr/local/bin/zramsync Signed-off-by: Markus Storm <markus.storm@gmx.net> * mount and rsync in /usr/local/bin/zramsync [skip ci] Signed-off-by: Markus Storm <markus.storm@gmx.net> * fix recursion error in /usr/local/bin/zramsync [skip ci] Signed-off-by: Markus Storm <markus.storm@gmx.net> * moved zramsync into zram-config.service as Pre/Post cmds Signed-off-by: Markus Storm <markus.storm@gmx.net> * moved zramsync into zram-config.service as Pre/Post cmds Signed-off-by: Markus Storm <markus.storm@gmx.net> * unattended Amanda install Signed-off-by: Markus Storm <markus.storm@gmx.net> * shellchecked Signed-off-by: Markus Storm <markus.storm@gmx.net> * revert oldish changes Signed-off-by: Markus Storm <markus.storm@gmx.net> * use zram-config.service from openhabian-zram repo Signed-off-by: Markus Storm <markus.storm@gmx.net> * use zramsync from openhabian-zram repo Signed-off-by: Markus Storm <markus.storm@gmx.net> * refactored amanda_setup to be able to run unattended Signed-off-by: Markus Storm <markus.storm@gmx.net> * add zramsync install back [skip ci] Signed-off-by: Markus Storm <markus.storm@gmx.net> * zram binaries bin -> sbin [skip ci] Signed-off-by: Markus Storm <markus.storm@gmx.net> * openhabian.conf.dist [skip ci] Signed-off-by: Markus Storm <markus.storm@gmx.net> * force image build Signed-off-by: Markus Storm <markus.storm@gmx.net> * only unattended Amanda install if backupdrive is set in openhabian.conf Signed-off-by: Markus Storm <markus.storm@gmx.net> * shellchecked Signed-off-by: Markus Storm <markus.storm@gmx.net>

mstormi · 2020-08-15T13:55:57Z

Ok this was quite some twiddling but I think I made it.
Please get yourself an external card reader and an SD card at least twice the size of the internal card and test this feature by activating any of the storageXXX options in (/boot/)openhabian.conf before running an unattended installation.
I've deployed this PR to the testbuild branch so point there in openhabian.conf.
Eventually you can also use the 50 menu options afterwards.

functions/helpers.bash

functions/backup.bash

functions/system.bash

ecdye · 2020-08-16T01:56:41Z

includes/sdrsync.service

+Type=oneshot
+#User=root
+#Group=root
+ExecStart=source /opt/openhabian/functions/backup.bash; mirror_SD "diff" %DEST


Does that work?

Frankly, I don't know yet

it didn't :) but now that I fixed it I think it does

functions/backup.bash

docs/NEWSLOG.md

openhabian-setup.sh

functions/backup.bash

ecdye

LGTM, however I do not currently have the ability to test, and would really prefer to have at least a little more testing before merging. @holgerfriedrich any thoughts?

holgerfriedrich

@ecdye @mstormi
Sorry, no time for a full review. Though, pls find a few of my thoughts below.

functions/backup.bash

holgerfriedrich · 2020-08-21T21:14:13Z

functions/backup.bash

+    if [[ -z "$retval" ]]; then return 0; fi
+    dest="/dev/$retval"
+  else
+    dest="${backupdrive:-/dev/sda}"


defaulting to /dev/sda could terribly go wrong, just thinking about my local linux box....

did you think of adding a few more checks, e.g. src!=dest, dest not mounted (which would also prohibit overwriting your own root partition....

You're right, I added a couple of checks and removed the default.
Better?

holgerfriedrich · 2020-08-21T21:18:15Z

functions/backup.bash

+  fi
+  if [[ "$1" == "raw" ]]; then
+    echo "Creating a raw partition copy, be prepared this may take long such as 20-30 minutes for a 16 GB SD card"
+    if ! cond_redirect dd if="${src}" bs=1M of="${dest}"; then echo "FAILED (raw device copy)"; return 1; fi


Markus, as we already discussed, I am concerned about the internal structure of the file system after copy. Inode tables etc of the source file system change during copy and this will always be a high risk to get inconsistent data or a (internally) broken file system.... You may not notice this on a first glance. Can you at least try to safeguard (fschk after copy, rsync again after this)?

I added a fsck.

holgerfriedrich · 2020-08-21T21:23:23Z

functions/backup.bash

+    dest="/dev/${retval}"
+  else
+    # shellcheck disable=SC2154
+    dest="${backupdrive:-/dev/sda}"


see comment below, I think /dev/sda is dangerous default value

same as below, any thoughts on checking dest before wiping the partion table? (e.g. dest is not the drive your normal installation runs on, dest has no currently mounted partitions, etc... or dest << 128GB (to avoid overwriting normal disks)

* implement ext card reader selection based on lsblk Signed-off-by: Markus Storm <markus.storm@gmx.net> * add mirrordrive boot parameter Signed-off-by: Markus Storm <markus.storm@gmx.net> * shellchecked Signed-off-by: Markus Storm <markus.storm@gmx.net> * add mount unit for /backup Signed-off-by: Markus Storm <markus.storm@gmx.net> * add mount unit for /backup Signed-off-by: Markus Storm <markus.storm@gmx.net> * add mirror setup during unattended install Signed-off-by: Markus Storm <markus.storm@gmx.net> * shellchecked Signed-off-by: Markus Storm <markus.storm@gmx.net> * add zramsync.service Signed-off-by: Markus Storm <markus.storm@gmx.net> * add zramsync.service Signed-off-by: Markus Storm <markus.storm@gmx.net> * first version mirror_SD Signed-off-by: Markus Storm <markus.storm@gmx.net> * first version mirror_SD [skip ci] Signed-off-by: Markus Storm <markus.storm@gmx.net> * first version mirror_SD [skip ci] Signed-off-by: Markus Storm <markus.storm@gmx.net> * first version mirror_SD [skip ci] Signed-off-by: Markus Storm <markus.storm@gmx.net> * first version mirror_SD [skip ci] Signed-off-by: Markus Storm <markus.storm@gmx.net> * define space for use by Amanda [skip ci] Signed-off-by: Markus Storm <markus.storm@gmx.net> * mount and rsync in /usr/local/bin/zramsync Signed-off-by: Markus Storm <markus.storm@gmx.net> * mount and rsync in /usr/local/bin/zramsync [skip ci] Signed-off-by: Markus Storm <markus.storm@gmx.net> * fix recursion error in /usr/local/bin/zramsync [skip ci] Signed-off-by: Markus Storm <markus.storm@gmx.net> * moved zramsync into zram-config.service as Pre/Post cmds Signed-off-by: Markus Storm <markus.storm@gmx.net> * moved zramsync into zram-config.service as Pre/Post cmds Signed-off-by: Markus Storm <markus.storm@gmx.net> * unattended Amanda install Signed-off-by: Markus Storm <markus.storm@gmx.net> * shellchecked Signed-off-by: Markus Storm <markus.storm@gmx.net> * revert oldish changes Signed-off-by: Markus Storm <markus.storm@gmx.net> * use zram-config.service from openhabian-zram repo Signed-off-by: Markus Storm <markus.storm@gmx.net> * use zramsync from openhabian-zram repo Signed-off-by: Markus Storm <markus.storm@gmx.net> * refactored amanda_setup to be able to run unattended Signed-off-by: Markus Storm <markus.storm@gmx.net> * add zramsync install back [skip ci] Signed-off-by: Markus Storm <markus.storm@gmx.net> * openhabian.conf.dist [skip ci] Signed-off-by: Markus Storm <markus.storm@gmx.net> * force image build Signed-off-by: Markus Storm <markus.storm@gmx.net> * only unattended Amanda install if backupdrive is set in openhabian.conf Signed-off-by: Markus Storm <markus.storm@gmx.net> * shellchecked Signed-off-by: Markus Storm <markus.storm@gmx.net> * always set initial backup shell+passwd [skip ci] Signed-off-by: Markus Storm <markus.storm@gmx.net> * no whitail in non-interactive install Signed-off-by: Markus Storm <markus.storm@gmx.net> * speedup mount units (no separate daemon-reload) Signed-off-by: Markus Storm <markus.storm@gmx.net> * fine select dev/part to mirror Signed-off-by: Markus Storm <markus.storm@gmx.net> * fine select dev/part to mirror Signed-off-by: Markus Storm <markus.storm@gmx.net> * shellchecked Signed-off-by: Markus Storm <markus.storm@gmx.net> * update NEWS with Auto-backup Signed-off-by: Markus Storm <markus.storm@gmx.net> * minor optimization of constants [skip ci] Signed-off-by: Markus Storm <markus.storm@gmx.net> * changes on review Signed-off-by: Markus Storm <markus.storm@gmx.net> * review,shellcheck Signed-off-by: Markus Storm <markus.storm@gmx.net> * do not run Amanda config generation twice Signed-off-by: Markus Storm <markus.storm@gmx.net> * auto backup docs [skip ci] Signed-off-by: Markus Storm <markus.storm@gmx.net> * [skip ci] remove test artifact Signed-off-by: Markus Storm <markus.storm@gmx.net> * add safety checks Signed-off-by: Markus Storm <markus.storm@gmx.net> * shellchecked Signed-off-by: Markus Storm <markus.storm@gmx.net>

mstormi · 2020-08-22T09:12:25Z

Would love you to test this with real HW and get me feedback before we merge.
Attach your card reader to your Pi and put in a larger SD card for a test.
I've created a build to contain this in the "testbuild" branch in #1098 .

Signed-off-by: Markus Storm <markus.storm@gmx.net>

…ed /storage Signed-off-by: Markus Storm <markus.storm@gmx.net>

Signed-off-by: Markus Storm <markus.storm@gmx.net>

mstormi · 2020-08-29T21:23:23Z

did anyone test ?
Please do, use this testbuild branch image

mstormi marked this pull request as draft August 5, 2020 22:26

mstormi force-pushed the bootbackup branch from 9acf1f2 to c23d21e Compare August 11, 2020 16:45

mstormi mentioned this pull request Aug 12, 2020

not syncing on ZRAM shutdown #1078

Closed

mstormi force-pushed the bootbackup branch 5 times, most recently from c6aa864 to 6554956 Compare August 14, 2020 10:31

ecdye mentioned this pull request Aug 15, 2020

debug log findings on install #1092

Closed

mstormi marked this pull request as ready for review August 15, 2020 13:23

mstormi requested review from ecdye and holgerfriedrich August 15, 2020 13:56

ecdye requested changes Aug 16, 2020

View reviewed changes

openhabian-setup.sh Outdated Show resolved Hide resolved

functions/backup.bash Show resolved Hide resolved

mstormi force-pushed the bootbackup branch from 9cb4a80 to f31ea1a Compare August 16, 2020 15:41

mstormi changed the title ~~mirror SD~~ auto backup Aug 16, 2020

mstormi requested a review from ecdye August 20, 2020 20:44

ecdye approved these changes Aug 21, 2020

View reviewed changes

holgerfriedrich reviewed Aug 21, 2020

View reviewed changes

mstormi added 23 commits August 29, 2020 10:57

use zramsync from openhabian-zram repo

a2eb117

Signed-off-by: Markus Storm <markus.storm@gmx.net>

refactored amanda_setup to be able to run unattended

c7bdca2

Signed-off-by: Markus Storm <markus.storm@gmx.net>

openhabian.conf.dist [skip ci]

a265ae1

Signed-off-by: Markus Storm <markus.storm@gmx.net>

force image build

a850f34

Signed-off-by: Markus Storm <markus.storm@gmx.net>

only unattended Amanda install if backupdrive is set in openhabian.conf

75af9bb

Signed-off-by: Markus Storm <markus.storm@gmx.net>

shellchecked

725bf5f

Signed-off-by: Markus Storm <markus.storm@gmx.net>

always set initial backup shell+passwd [skip ci]

18fdb46

Signed-off-by: Markus Storm <markus.storm@gmx.net>

no whitail in non-interactive install

b4b8605

Signed-off-by: Markus Storm <markus.storm@gmx.net>

speedup mount units (no separate daemon-reload)

c9afe8d

Signed-off-by: Markus Storm <markus.storm@gmx.net>

fine select dev/part to mirror

4e6494a

Signed-off-by: Markus Storm <markus.storm@gmx.net>

fine select dev/part to mirror

d3fa020

Signed-off-by: Markus Storm <markus.storm@gmx.net>

shellchecked

c1b09c2

Signed-off-by: Markus Storm <markus.storm@gmx.net>

update NEWS with Auto-backup

3af0a47

Signed-off-by: Markus Storm <markus.storm@gmx.net>

minor optimization of constants [skip ci]

4a683db

Signed-off-by: Markus Storm <markus.storm@gmx.net>

changes on review

376bf51

Signed-off-by: Markus Storm <markus.storm@gmx.net>

review,shellcheck

134c9cf

Signed-off-by: Markus Storm <markus.storm@gmx.net>

do not run Amanda config generation twice

34b1e56

Signed-off-by: Markus Storm <markus.storm@gmx.net>

auto backup docs [skip ci]

e521f47

Signed-off-by: Markus Storm <markus.storm@gmx.net>

[skip ci] remove test artifact

9840a1a

Signed-off-by: Markus Storm <markus.storm@gmx.net>

add safety checks

33dce88

Signed-off-by: Markus Storm <markus.storm@gmx.net>

shellchecked

25c2cff

Signed-off-by: Markus Storm <markus.storm@gmx.net>

install Amanda after eventual SD mirror so slots are created on mount…

4f0d1a4

…ed /storage Signed-off-by: Markus Storm <markus.storm@gmx.net>

parametrize %DEVICE = in storage.mount [skip ci]

770ecf6

Signed-off-by: Markus Storm <markus.storm@gmx.net>

mstormi force-pushed the bootbackup branch from 0efa949 to 770ecf6 Compare August 29, 2020 09:11

mstormi added 3 commits August 29, 2020 19:52

update NEWS.md

cb8142d

Signed-off-by: Markus Storm <markus.storm@gmx.net>

fix and enable SD sync timers

913a736

Signed-off-by: Markus Storm <markus.storm@gmx.net>

fix and enable SD sync timers

5f58646

Signed-off-by: Markus Storm <markus.storm@gmx.net>

mstormi merged commit c4d2e8e into openhab:master Aug 31, 2020

mstormi deleted the bootbackup branch March 7, 2021 17:37

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

auto backup #1077

auto backup #1077

mstormi commented Aug 5, 2020 •

edited

Loading

holgerfriedrich commented Aug 6, 2020

mstormi commented Aug 6, 2020 •

edited

Loading

mstormi commented Aug 6, 2020 •

edited

Loading

ecdye commented Aug 7, 2020

ecdye commented Aug 12, 2020

ecdye commented Aug 12, 2020

mstormi commented Aug 12, 2020 •

edited

Loading

mstormi commented Aug 15, 2020 •

edited

Loading

ecdye Aug 16, 2020

mstormi Aug 16, 2020

mstormi Aug 31, 2020

ecdye left a comment

holgerfriedrich left a comment

holgerfriedrich Aug 21, 2020

holgerfriedrich Aug 21, 2020

mstormi Aug 21, 2020 •

edited

Loading

holgerfriedrich Aug 21, 2020

mstormi Aug 21, 2020

holgerfriedrich Aug 21, 2020

holgerfriedrich Aug 21, 2020

mstormi Aug 21, 2020

mstormi commented Aug 22, 2020

mstormi commented Aug 29, 2020

auto backup #1077

auto backup #1077

Conversation

mstormi commented Aug 5, 2020 • edited Loading

holgerfriedrich commented Aug 6, 2020

mstormi commented Aug 6, 2020 • edited Loading

mstormi commented Aug 6, 2020 • edited Loading

ecdye commented Aug 7, 2020

ecdye commented Aug 12, 2020

ecdye commented Aug 12, 2020

mstormi commented Aug 12, 2020 • edited Loading

mstormi commented Aug 15, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ecdye left a comment

Choose a reason for hiding this comment

holgerfriedrich left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mstormi Aug 21, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mstormi commented Aug 22, 2020

mstormi commented Aug 29, 2020

mstormi commented Aug 5, 2020 •

edited

Loading

mstormi commented Aug 6, 2020 •

edited

Loading

mstormi commented Aug 6, 2020 •

edited

Loading

mstormi commented Aug 12, 2020 •

edited

Loading

mstormi commented Aug 15, 2020 •

edited

Loading

mstormi Aug 21, 2020 •

edited

Loading