Repository navigation
Replies: 15 comments 1 reply
|
10.2.0 has a step for exactly this, and you don't need to switch DXP480T S3 off for it. Open Storage locations → DXP480T S3. Its page has an Own folders card with a Use as own folder button. In the window, pick the sections that should move, leave Copy the backups already made on and press Switch. Before you switch, What happens shows what it will do for each section: DXP480T S3 already holds a copy, so that copy becomes the section's own folder and copying to DXP480T S3 ends. Any backups the copy doesn't have yet are copied over first. Mani and Qnap keep getting their copies as before, and DXP480T S3 stops being an off-site copy once the sections have moved. Two more things:
|
|
Done. |
|
After doing this, did a backup and got this: BUT: |
|
Thanks, that's a bug. Moving ZFS along is what brought it out: ZFS only replicates and never writes a backup, so nothing was ever created in its folder on DXP480T S3. The prune after each round took it for granted that a repository on S3 is there, asked restic to prune it, and restic said it doesn't exist. Your backups and the replica are fine. The red line is just that prune. 10.2.1 fixes it. Prune, verify, unlock and the restore check now skip a bucket folder that holds no repository yet, the way they already skip an empty folder on the server. About the button before you added DXP480T S3 as an off-site copy: it should work for a fresh storage location too, and copy the backups over first. Do you remember what the window showed? Was Switch greyed out, or was there a message under What happens? With 1.6 TB per section the window counts the backups first, which can take a while, and Switch stays off until that's done. |
|
The Switch was greyed out and not selectable. |
|
I readded the zfs datasets again. So why does it show Local?? 
And waiting for first backup? |
|
Thanks for the screenshots, those are two display bugs, and 10.2.2 fixes them. With Replica only, no backup switched on, an entry said "Waiting for the first backup" in the list and on its page, although no backup is ever coming. It now says "Waiting for the first replica" until the replica has run once, and counts as protected after that. Its page also listed the section's own folder ("Local", 0 backups) although that folder is off for it. The card now shows the replica and only the storage locations that still hold older backups of the entry. About the greyed-out Switch: what did it say under What happens at that moment, "Checking…", a line in orange, or the list of steps? Before the window lets you switch, it asks the server what the move would do, and the server first reads the backups on both sides for each section. With 1.6 TB of backups that can take a while, and Switch stays off until the answer is there. On a test instance with a fresh REST location the answer came back within a few seconds, so I'd like to know which of the three you saw. If you run into it again, a screenshot of the whole window would help. |
|
I remember now: the repo had been freshly created in rustfs and was empty. The button was greyed out and nothing was shown in What happens (as it was empty). |
|
Just did a Replicate all now. It only starts replication of first dataset instead of all 3. The rest I had to do manually. |
|
Another thing I find: |
|
10.3.0 is out and covers the rest of what you ran into here. Replicate all now runs the whole batch on the server, one entry after the other, whether the page stays open or not. An entry that can't replicate is named and skipped, and the others still run. The greyed-out Switch with the empty bucket: when the window's check failed, it showed nothing under What happens. Now the reason shows up there in a box, and when only copying the existing backups is in the way, it tells you that switching off Copy the backups already made lets the move go through. A move between two places that sign in differently, like your RustFS location with its own keys and the folder on the server, works too, because each side is read with its own sign-in. On speed: a replica to a ZFS server can take a shorter way. The host pipes zfs send straight into its own ssh to the target, with BombVault's replica key, and only goes through the container when it can't. ssh prefers AES-GCM, and a stream that does go through BombVault gets a 32 MiB buffer. I couldn't measure this on real hardware, so I can't promise you'll get near the numbers of the SpaceInvaderOne script. If you try it, I'd like to hear what rate you see. The rate BombVault shows counts the compressed stream, so keep that in mind when you compare. One more thing: switching Replica only, no backup off used to leave the off-site copies of the ZFS section switched off without a word. That's a setting of its own now. If you switched Replica only off at some point, check the ZFS switch on your storage locations' pages. |
|
"One more thing: switching Replica only, no backup off used to leave the off-site copies of the ZFS section switched off without a word. That's a setting of its own now. If you switched Replica only off at some point, check the ZFS switch on your storage locations' pages."........ Don't exactly understand what you mean.... 
|
|
This made a HUGE difference!!!! 
|
|
Improvement: the script also keeps the defined number of snapshots in the source (in addition to the target). This is another layer of protection and in an emergency it is quicker because you can just revert the snapshot on the target. I needed this yesterday. I had a new docker changing all the permissions on /mnt/user...... you can imagine the s** I was in.... |
|
6.6 Gbps, great to see that, and thanks for measuring it. Keeping snapshots on the source is in 10.3.1. The Replica card of a ZFS entry has a new field, Keep on this server. It sets how many of the newest replica snapshots stay on the source after each replica, on top of the ones the target keeps. Files in them can be read or copied right away from .zfs/snapshot or a clone, with nothing to transfer. Going back to the newest one is a plain zfs rollback. Rolling a dataset back to an older one takes a couple more steps: zfs release on the newest first (it keeps a hold, because the next transfer starts from it), then zfs rollback -r. The next replica then stops and names the snapshot to roll the copy on the target back to, the newest one both sides still share. Roll it back to that one and the replica carries on. BombVault never throws away snapshots on the target by itself. The manual has the exact commands. Sorry about the permissions mess, and glad the morning snapshot got you out of it. About the switch: in your setup there's nothing to do. Your off-site locations never had a ZFS copy, which is why they show no ZFS row. The ZFS folder on DXP480T S3 is the section's own folder, and with every entry on Replica only nothing gets written there, so it stays empty. The note was meant for people who had ZFS off-site copies and switched Replica only on and off again. |
Thanks a lot!!!! retiring scripts now...... |
Uh oh!
There was an error while loading. Please reload this page.
what is the best/correct way to change the main (local) backup to one on a s3 on another unraid?
I have created the new s3 already in storage locations. I also have added it (temporarily to already receive backups) as another offsite:
I intend of disabling "DXP480T S3" after the 1st offsite upload.
But then....?
All reactions