Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

implemented creation of zfs bookmarks #342

Merged
merged 5 commits into from Mar 29, 2019

Conversation

Projects
None yet
3 participants
@phreaker0
Copy link
Collaborator

commented Jan 30, 2019

Fixes #336

phreaker0 added some commits Jan 30, 2019

@mailinglists35

This comment has been minimized.

Copy link

commented Feb 1, 2019

Hi,
finally getting some time to look at it

first remark: I am getting permission denied when doing initial send (no send/recv options used yet) remote to remote - I know, I am the developers pain in the a**, finding all corner cases :)

I will attempt to see if github lets me comment directly on your PR

will comment if/when I discover more

Show resolved Hide resolved syncoid Outdated
@mailinglists35

This comment has been minimized.

Copy link

commented Feb 2, 2019

I have one more case :)

Let's have this:

no target
[suport@zfspv ~]$ zfs get -r guid bkppv/encrypted/bkphq/delete cannot open 'bkppv/encrypted/bkphq/delete': dataset does not exist

source with one manual snapshot
[suport@zfshq ~]$ zfs get guid -r bkphq/delete NAME PROPERTY VALUE SOURCE bkphq/delete guid 18092307491229082710 - bkphq/delete@firstsync guid 2836662525218684215 -
invoke first sync with --debug --create-snapshot --no-sync-snap https://pastebin.com/aYESSPiU
which will create target and send from the only source snapshot, then will create bookmark

invoke a second time, and this time (output included in pastebin above too) after "INFO: no snapshots on source newer than firstsync on target. Nothing to do, not syncing.", it attempts to create the bookmark, but it fails because it has the same name.

I was wondering if wether
should this give up creating bookmark after it finds identical latest snapshot on source and target, or should it delete the matching bookmark and create it again to match the matching snapshot?

but then I thought that later in time we can be in the following situation:
original matching snapshot can get deleted at source, but later a new one created with the same name

So neither recreating nor abandoning creation would be optimal, but creating the bookmark with an additional suffix in the name (for example -$(seconds since midnight padded with zeroes to be a fixed length)-$(random 3 digits) - i could be wrong but that's the first that i imagined EDIT: or first field of cksum coreutils command with the snapshot's guid at input - zfs get guid -o value -H bkphq/delete@firstsync | cksum | cut -d' ' -f1 ).

Since when searching matching pairs we are comparing guids, this will not affect finding the matching bookmark-snapshot pair, but will allow creating new bookmark even if the snapshot name is the same but taken at different time with a different guid than the original one. I would not delete the bookmark since it might be useful when replicating to a second or third target.

What do you think?

@mailinglists35

This comment has been minimized.

Copy link

commented Feb 2, 2019

if that complicates things too much or takes too much of your time, i'd be happy even with random bookmark name instead, ie:

$bookmarkcmd = "$sourcesudocmd $zfscmd bookmark $sourcefsescaped\@$newsyncsnapescaped $sourcefsescaped\#$<guid of $newsyncsnap>";

@mailinglists35

This comment has been minimized.

Copy link

commented Feb 2, 2019

oh well, I think both take about the same time and effort... since in both cases you'd do an additional $bookname=$( something ) then use $bookname instead of $newsyncsnapescaped in $bookmarkcmd

@mailinglists35

This comment has been minimized.

Copy link

commented Feb 2, 2019

even with pseudo unique bookmark names we still must perform an additional check when creating, so we do not attempt to create the same one again. i'm trying to look around $bookmarkcmd to write a PR with this verify

@mailinglists35

This comment has been minimized.

Copy link

commented Feb 2, 2019

sorry, i've done quite a mess both in perl and github but it works for me now.
I hope your perl knowledge can transform my miserable coding into a better one :)

phreaker0#2

phreaker0 added some commits Feb 4, 2019

@mailinglists35

This comment has been minimized.

Copy link

commented Feb 11, 2019

I got this running and it works pretty well - thank you so much!

I've looked at some cases where source and target were linked by just one single common snapshot while each having lots of other snapshots; the sync process sent lots of intermediary snapshots, but the bookmark creation was done only for the newest snapshot.

Is it possible to create bookmarks for each intermediary snapshot that is found to be between oldest snapshot and newest snapshot? This would greatly increase future chances of falling back to a bookmark by having more bookmarks available at source while allowing more aggressive snapshot cleanup on target.

Thank you again and have a great day!

@phreaker0

This comment has been minimized.

Copy link
Collaborator Author

commented Feb 13, 2019

@mailinglists35 this can be implemented in a follow up PR

@jimsalterjrs jimsalterjrs merged commit e376ead into jimsalterjrs:master Mar 29, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.