Provide example setup to get async netrw-write for remotes even when using :w

[kiryph]

When editing a remote file, saving the file with :w is blocking the editor until the file is saved to the remote. This can vary dependent on file size and remote connection.

Would it be possible to offer a setup in the wiki similar to the fugitive setting of :Gpush to make this non-blocking?

netrw-write acts on the builtin write command :write via the autocommand events BufWriteCmd and FileWriteCmd as can be seen in $VIMRUNTIME/plugin/netrwPlugin.vim:

 au BufWriteCmd  ftp://*,rcp://*,scp://*,http://*,file://*,dav://*,davs://*,rsync://*,sftp://*          exe "sil doau BufWritePre ".fnameescape(expand("<amatch>"))|exe 'Nwrite '.fnameescape(expand("<amatch>"))|exe "sil doau BufWritePost ".fnameescape(expand("<amatch>"))
 au FileWriteCmd ftp://*,rcp://*,scp://*,http://*,file://*,dav://*,davs://*,rsync://*,sftp://*          exe "sil doau FileWritePre ".fnameescape(expand("<amatch>"))|exe "'[,']".'Nwrite '.fnameescape(expand("<amatch>"))|exe "sil doau FileWritePost ".fnameescape(expand("<amatch>"))

[skywind3000]

unfortunately, all block works are done in the command Nwrite, there is not any single interface in netrw to execute shell commands. it appears that there are half of the netrw code need to be updated to support AsyncRun or async-job directly in vim 8.

[kiryph]

Sorry, for a possibly stupid suggestion: A simple insertion of AsyncRun command into the given code lines from $VIMRUNTIME/plugin/netrwPlugin.vim won't do the trick?

That would be too bad, if there is no possibility without a major code rewrite of netrw which will presumably never happen.

[kiryph]

Ok, I should read your plugin description properly: Run Async Shell Commands in Vim 8.0 and Output to Quickfix Window

So I checked netrw again: netrw uses a central function fun~ s:NetrwExe(cmd) which is used by all other methods (Nwrite, Nsource, ...) which looks like

" s:NetrwExe: executes a string using "!" {{{2
fun! s:NetrwExe(cmd)
"  call Dfunc("s:NetrwExe(a:cmd)")
  if has("win32") && &shell !~? 'cmd' && !g:netrw_cygwin
    let savedShell=[&shell,&shellcmdflag,&shellxquote,&shellxescape,&shellquote,&shellpipe,&shellredir,&shellslash]
    set shell& shellcmdflag& shellxquote& shellxescape&
    set shellquote& shellpipe& shellredir& shellslash&
    exe a:cmd
    let [&shell,&shellcmdflag,&shellxquote,&shellxescape,&shellquote,&shellpipe,&shellredir,&shellslash] = savedShell
  else
   exe a:cmd
  endif
"  call Dret("s:NetrwExe")
endfun

The scp call from fun! netrw#NetWrite(...) range is

call s:NetrwExe(s:netrw_silentxfer."!".g:netrw_scp_cmd.useport." ".s:ShellEscape(tmpfile,1)." ".s:ShellEscape(g:netrw_machine.":".b:netrw_fname,1))

A hacky way to do it would be by simply replacing in the command string the ! with AsyncRun:

" s:NetrwExe: executes a string using "!" {{{2
fun! s:NetrwExe(cmd)
"  call Dfunc("s:NetrwExe(a:cmd)")
  if has("win32") && &shell !~? 'cmd' && !g:netrw_cygwin
    let savedShell=[&shell,&shellcmdflag,&shellxquote,&shellxescape,&shellquote,&shellpipe,&shellredir,&shellslash]
    set shell& shellcmdflag& shellxquote& shellxescape&
    set shellquote& shellpipe& shellredir& shellslash&
    exe a:cmd
    let [&shell,&shellcmdflag,&shellxquote,&shellxescape,&shellquote,&shellpipe,&shellredir,&shellslash] = savedShell
  else
   exe substitute(a:cmd, "!", "AsyncRun ", "")
   " exe a:cmd
  endif
"  call Dret("s:NetrwExe")
endfun

[skywind3000]

Cool, does it work ?

[skywind3000]

sometimes, netrw may do something which relies on a command's end,
Is it a little dangerous to replace all the bang to "AsyncRun" ?

[kiryph]

It looks promising but has flaws

• When opening files with $ vim scp://... you have to enter a second time the open command in vim with :e scp://....
• Writing could work but there is a problem with the arguments. :w runs AsyncRun but fails with:

|| [scp -q '/var/folders/d_/5kjxh9v92k752bvzf559fd3c0000gn/T/vXx8bYF/7' 'user@remote:results.txt']
|| /var/folders/d_/5kjxh9v92k752bvzf559fd3c0000gn/T/vXx8bYF/7: No such file or directory
|| [Finished in 0 seconds with code 1]

The correct temp filename is '/var/folders/d_/5kjxh9v92k752bvzf559fd3c0000gn/T/vXx8bYF/7.txt'

I suspect there is a problem with s:ShellEscape(tmpfile,1).

[skywind3000]

is it possible that netrw itself deletes that temp file ? because it believes scp was finished after a s:NetrwExe invoking ?
! will wait until scp is over but AsyncRun will return immediately .

If so, maybe a refactor is needed to support async jobs

[kiryph]

I think you are right. There is a cleanup in fun! netrw#NetWrite(...) range which deletes the file:

  " NetWrite: Cleanup: {{{3
"  call Decho("cleanup",'~'.expand("<slnum>"))
  if s:FileReadable(tmpfile)
"   call Decho("tmpfile<".tmpfile."> readable, will now delete it",'~'.expand("<slnum>"))
   call s:NetrwDelete(tmpfile)
  endif

So the cleanup has to be moved to AsyncRun -post which requires more changes to netrw than I was hoping for.

I am only interested in an async :w which could be realized by

• modifying fun! netrw#NetWrite(...) range (no file deletion and a call to a new fun! s:NetrwExeAndPost(cmd, post_vim_function))
• addingfun! s:NetrwExeAndPost(cmd, post_vim_function) with AsyncRun -post=s:NetrwDelete(tmpfile)

I will see how this works out.

[skywind3000]

the right form of

AsyncRun -post=s:NetrwDelete(tmpfile) ...

should be

AsyncRun -post=call\ s:NetrwDelete(tmpfile) ...

good luck !

[kiryph]

Thanks for your reminder that a call has to be inserted.

However, AsyncRun does not know the s:netrwDelete function:

E117: Unknown function: <SNR>84_NetrwDelete 

Apparently, this is an issue of the s: scope. I fixed this with using vim's delete() function.

Following diff to $VIMRUNTIME/autoload/netrw.vim (version 156) saves asynchronously with AsyncRun when you put into your vimrc let g:netrw_write_AsyncRun = 1:

❯ git diff netrw-156.vim netrw.vim
diff --git a/netrw-156.vim b/netrw.vim
index 76485c2..183fc96 100644
--- a/netrw-156.vim
+++ b/netrw.vim
@@ -510,6 +510,7 @@ call s:NetrwInit("g:NetrwTopLvlMenu","Netrw.")
 call s:NetrwInit("g:netrw_win95ftp",1)
 call s:NetrwInit("g:netrw_winsize",50)
 call s:NetrwInit("g:netrw_wiw",1)
+call s:NetrwInit("g:netrw_write_AsyncRun",0)
 if g:netrw_winsize > 100|let g:netrw_winsize= 100|endif
 " ---------------------------------------------------------------------
 " Default values for netrw's script variables: {{{2
@@ -2377,6 +2378,14 @@ fun! netrw#NetWrite(...) range
 "    call Decho("(netrw) Processing your write request...",'~'.expand("<slnum>"))
    endif
+   " NetWrite: Perform AsyncRun Write {{{3
+   " ============================
+   if exists("g:netrw_write_AsyncRun") && g:netrw_write_AsyncRun == 1
+       let bang_cmd = 'AsyncRun -post=call\ delete('.s:ShellEscape(tmpfile,1).')\ |\ echo\ "(netrw)\ Your\ write\ request\ has\ finished." '
+    else
+        let bang_cmd ="!"
+   endif
+
    ".........................................
    " NetWrite: (rcp) NetWrite Method #1 {{{3
    if  b:netrw_method == 1
@@ -2515,7 +2524,7 @@ fun! netrw#NetWrite(...) range
     else
      let useport= ""
     endif
-    call s:NetrwExe(s:netrw_silentxfer."!".g:netrw_scp_cmd.useport." ".s:ShellEscape(tmpfile,1)." ".s:ShellEscape(g:netrw_machine.":".b:netrw_fname,1))
+    call s:NetrwExe(s:netrw_silentxfer.bang_cmd.g:netrw_scp_cmd.useport." ".s:ShellEscape(tmpfile,1)." ".s:ShellEscape(g:netrw_machine.":".b:netrw_fname,1))
     let b:netrw_lastfile = choice

    ".........................................
@@ -2612,9 +2621,11 @@ fun! netrw#NetWrite(...) range

   " NetWrite: Cleanup: {{{3
 "  call Decho("cleanup",'~'.expand("<slnum>"))
-  if s:FileReadable(tmpfile)
-"   call Decho("tmpfile<".tmpfile."> readable, will now delete it",'~'.expand("<slnum>"))
-   call s:NetrwDelete(tmpfile)
+  if !exists("g:netrw_write_AsyncRun") || g:netrw_write_AsyncRun == 0
+    if s:FileReadable(tmpfile)
+"     call Decho("tmpfile<".tmpfile."> readable, will now delete it",'~'.expand("<slnum>"))
+      call s:NetrwDelete(tmpfile)
+    endif
   endif
   call s:NetrwOptionRestore("w:")

There might be issues under windows which s:NetrwDelete handles. Also the check whether the file is readable is not applied for post delete.

[skywind3000]

great job !
I even thought it was impossible to patch netrw without the help of the author.

[kiryph]

Thanks for your compliment. I certainly agree that netrw is a monster plugin with an intimidating plugin file of 12000 lines of code. Vim with syntax highlighting and folding chokes quite a bit when opening this monster. However, the code is readable and structured which makes it possible to find your way around, in particular with folding enabled.

Keep in mind, this patch is not comprehensive. It works for me under OSX using scp. Extending netrw properly is more work and can probably only be done by the author. Since this plugin is already quite large I am not sure whether the author wants to add a possibility to override the bang command as I did. Implementing vim8 async support in netrw is even more work.

So I am not sure how I want to proceed here. I am happy right now with my solution but I think it is not good enough to be a general solution one would point someone to with the same question. What are your thoughts?

[skywind3000]

I wrote a wiki page to recommend this netrw patch to people who may be interested in this topic.
and they must take their own risk.

The best solution is suggesting the author to support async jobs directly. After all, netrw is a default plugin in the vim runtime, which will be widely distributed without asyncrun.

This patch can be a good option if the author has no time to add async feature to netrw.

[kiryph]

I actually contacted Charles Campbell aka DrChip

Dear Chip Campbell,

with vim8 available quite a few plugins for async execution in vim
have been developed.

One of them is https://github.com/skywind3000/asyncrun.vim.

What do you think about following rough patch to allow for async saves
using this plugin:

...

His response is (hopefully, he doesn't mind publishing it here)

Hello:

I'll keep this in mind, but my near term efforts with netrw will be to remove some bugs.
I will look into it, though.

Thank you,
Chip Campbell