Send initial data to destination in separate step

[foxycode]

Currently when I initialize new dataset, it is synced to destionation right away. When network error happens, this process is broken. I'd liek to initialize dataset without initial transfer and let that happen by cron on first sync.

[ppbrown]

I'm not sure what you are suggesting here. Please be more specific? It's not making sense to me.
What part do you want to "initialize", and how?
To my mind, first "initialize" is done by initial transfer. It's the first call to the remote side if I recall.
The zfs send/receive pair?
So "initial transfer", is "initialize dataset", in my perspective.

[foxycode]

I'd like to skip remote part completely or stay with zfs create only when initializing new dataset to avoid waiting for sync to be done when I am on SSH console. Yes, I mean zfs send/recv part.

[ppbrown]

It's been a while since I've dug deeply into this part of my code, so.. correct me if I'm missing something here, but...

you're requesting a code change... to a part of the code that you will never use again, once I get around to implementing your OTHER request to use recv -u.

Right?
:-D

convenience link: issue #38

[ppbrown]

Hmm.. you make reference to an odd thing, "SSH console".
And I smell potential reference to other things you havent mentioned.
So, maybe the part you left out is that you have some special plan or framework to manually do zfs send/receive outside of zrep?

Maybe you should tell me more about that. The more I know, the better a potential solution will be.

[foxycode]

Maybe workflow I am trying to implement will tell you more.

On server I want to backup:

1. I join to server using SSH
2. I call zrep init <dataset> ... - no send/recv here, because it will take lot of time
3. Cron will do initial send/recv when zrep sync all is called

I simply don't want to do initial data transfer, it takes huge amount of time on my servers and it can easily break when my connection with server is interrupted. It has nothing to do with -u flag.

Hope it is clearer now.

[ppbrown]

that is not a valid workflow.
Yes, the initial sync takes time. That's unavoidable, because you have to copy the entire dataset over!

"zrep sync all", can only work, AFTER a full send has been done. "sync" is incremental-only.
Incrementals can only work, after you have done a FULL send, aka "init".

I'm .. surprised.. that you find use of cron, more favourable than use of command line. But thats okay, there's a solution for people like you ;)
Look into the "at" command. Its just like cron, except one-time use. So, create an 'at' job, for the
"zrep init"?

you can do something like
"at now zrep init blah blah blah"

then drop your ssh connection, but the system will run the init in the background just like a cronjob would.

OTHER people, might instead do

#  nohup zrep init blahblah &
#  exit

which under most shells will accomplish pretty much the same thing.

[foxycode]

Never mind, looks liek we don't understand each other.

And thanks for the at tip, didn't know that.

[ppbrown]

PS: there's also other solutions for "my network sucks".
Run a terminal multiplexor like "screen".
There are lots of fun tricks you can do with screen, but the most basic is protecting your shell session from dumb network problems.
It goes like this.

When you ssh into a box, start up "screen". It then gives you a virtualized session.

ssh remotehost
screen
bash-4.1$ while true; do date; sleep 5 ; done  
Mon Feb 27 15:13:48 PST 2017
Mon Feb 27 15:13:53 PST 2017
Mon Feb 27 15:13:58 PST 2017
Mon Feb 27 15:14:03 PST 2017

(kill ssh with ~ .)

ssh remotehost
screen -r  
### reconnects to your saved session, and scrolls the output that you missed
### the command has continued running uninterrupted...
Mon Feb 27 15:14:03 PST 2017
Mon Feb 27 15:14:08 PST 2017
Mon Feb 27 15:14:13 PST 2017
Mon Feb 27 15:14:18 PST 2017
Mon Feb 27 15:14:23 PST 2017
Mon Feb 27 15:14:28 PST 2017
...