Can't find temp file when call VimtexClean

[vanabel]

I know this is an old bug here, but I find something new that I can't understand here. I try to fix the error when I run :VimtexClean, which will popup an error message:

E484: Can't open file C:\Users\usrname\AppData\Local\Temp\VIoF1B7.tmp

The problem is that the VIoF1B7.tmp can be found at the exactly location, and can be deleted without any permission.

While debugging, I finally find that this line causes the problem:

silent call system(self.prepared_cmd)

If I comment it completely, all thing gone (of course no clean at all). It seems strange to me that the self.prepared_cmd is

start /b cmd /s /c "cd /D "C:\Users\usrname\Desktop" & latexmk -c  -outdir=C:\Users\usrname\vimfiles\myvim\vimtemp "test.tex" >nul"

and the self discretionary is:

[['pid', 0], ['background', 1], ['continuous', 0], ['win32_restore_shell', 0], ['cmd', 'cd /D "C:\Users\usrname\Desktop" & latexmk -c  -outdir=C:\Users\usrname\vimfiles\myvim\vimtemp "test.tex"'], ['stop', function('5084')], ['_execute', function('5088')], ['workdir', ''], ['get_pid', function('5092')], ['silent', 1], ['_do_not_run', function('5086')], ['pprint_items', function('5085')], ['prepared_cmd', 'start /b cmd /s /c "cd /D "C:\Users\usrname\Desktop" & latexmk -c  -outdir=C:\Users\usrname\vimfiles\myvim\vimtemp "test.tex" >nul"'], ['_pre_run', function('5087')], ['output', 'null'], ['_post_run', function('5089')], ['run', function('5083')], ['_prepare', function('5090')], ['_restore', function('5091')]]

I learnt from this fix, that the VimtexClean call can be fixed by surround the l:cmd with double quote, while this fix violate the sumartpdf previewer call. So I think a if statement will work for this:

function! s:process._execute() abort dict " {{{1
  if self.silent
	echo items(self)
    if self.output == 'null'
		let self.prepared_cmd = '"' . self.prepared_cmd . '"'
	endif
    silent call system(self.prepared_cmd)
  elseif self.background
    silent execute '!' . self.prepared_cmd
    if !has('gui_running')
      redraw!
    endif
  else
    execute '!' . self.prepared_cmd
  endif

MY Question: I don't understand why there always a new temp file created whenever I call either VimtexCompile or VimtexClean, why we get such an error while the temp file exists?

[lervag]

I know this is an old bug here, but I find something new that I can't understand here.

I do not have a perfect memory, so for me it is difficult to know that this is an old bug. If you have references to related, old issues, please add them explicitly.

I learnt from this fix, that the VimtexClean call can be fixed by surround the l:cmd with double quote, while this fix violate the sumartpdf previewer call. So I think a if statement will work for this:

I assume you mean "SumartraPDF" here, right? Also, for your suggested fix with if, please show the diff. It is not clear to me which part was added by you and which part is vimtex (unless I explicitly check it). By making things like this more clear, you help me to understand what you are suggesting, which saves time and effort on my side. :)

MY Question: I don't understand why there always a new temp file created whenever I call either VimtexCompile or VimtexClean, why we get such an error while the temp file exists?

:VimtexCompile uses a temporary file by default in order for us to be able to inspect the output (with :VimtexOutput). :VimtexClean should not write to a temporary file, instead, it should write to the /dev/null (or nul) file handle, i.e., discard all output. I am not sure why the temporary file gets involved in this case. However, I'm no Windows expert, and it might be something that happens as a side effect of system(). I searched and found a similar issue, tpope/vim-dispatch#108, where it is suggested to use set noshelltemp. Does that work? (Try to add it either in your vimrc file or in s:process._prepare() similar to the other options that are temporarily toggled.)

---

Finally, I took the liberty of cleaning your issue post by removing unused parts of the issue template. Please ensure that your issues are properly formated and that you only keep the relevant parts*. This makes things much less confusing for me and makes it easier to help.

Note: If you intended to keep those parts(*), then you should update the text so that it is clear that those parts were part of your description and not just part of the template.

[vanabel]

I am sorry for the unclear in my issue and thanks for your suggestion of the improvement. I hope the following will be more clear to address my issue and my question, however, if not, please be more patient to me, since I just want to provide a solution to the problem I encountered.

1. The old bug I mean is issued here Extra quotes while preparing commands to run in windows #1059
   In that issue, I find the fix, which add a double quote of the command, however, it broken the \lv command for SumartPDF viewer.

So I investigate the issue (it takes my whole day to read your vimscript, since I don't know how to debug in vim, and the function definition with dict, the self variable..., and finally locate the exactly bug line:silent call system(self.prepared_cmd)). I want to provide a NOT perfect at all fix. The idea is quit simple:

2. The previous fix works for :VimtexClean but broken :VimtexCompile, so I just add a IF statement to distinguish them, the diff is given below (I am sorry again for the unclear in my old post):

$ git diff
diff --git a/autoload/vimtex/process.vim b/autoload/vimtex/process.vim
index a21b410..09b3ade 100644
--- a/autoload/vimtex/process.vim
+++ b/autoload/vimtex/process.vim
@@ -100,6 +100,10 @@ endfunction
 " }}}1
 function! s:process._execute() abort dict " {{{1
   if self.silent
+       "echo items(self)
+    if self.output == 'null'
+               let self.prepared_cmd = '"' . self.prepared_cmd . '"'
+       endif
     silent call system(self.prepared_cmd)
   elseif self.background
     silent execute '!' . self.prepared_cmd

3. For my question, I want to provide the command called that when I call :VimtexClean and :VimtexCompile to help a further investigation. The code I changed for output the self.prepared_cmd:

$ git diff
diff --git a/autoload/vimtex/process.vim b/autoload/vimtex/process.vim
index a21b410..b3df93f 100644
--- a/autoload/vimtex/process.vim
+++ b/autoload/vimtex/process.vim
@@ -100,7 +100,8 @@ endfunction
 " }}}1
 function! s:process._execute() abort dict " {{{1
   if self.silent
-    silent call system(self.prepared_cmd)
+       echo self.prepared_cmd
+       silent call system(self.prepared_cmd)
   elseif self.background
     silent execute '!' . self.prepared_cmd
     if !has('gui_running')

and the output when I call :VimtexClean is

start /b cmd /s /c "cd /D "C:\Users\van\Desktop" & latexmk -c  -outdir=C:\Users\van\vimfiles\myvim\vimtemp "test.tex" >nul"
Error detected while processing function vimtex#compiler#clean[1]..1430[15]..vimtex#process#run[5]..1675[5]..1680:
line    6:
E484: Can't open file C:\Users\van\AppData\Local\Temp\VIoAEA6.tmp

The problem is that the file VIoAEA6.tmp is created at the exact location, and I can delete with !del C:\Users\van\AppData\Local\Temp\VIoAEA6.tmp, so it is not the permission problem.

The output of :VimtexCompile is

start /b cmd /s /c "set max_print_line=2000 & latexmk -verbose -file-line-error -synctex=1 -interaction=nonstopmode -pdf -outdir=C:\Users\van\vimfiles\myvim\vimtemp -pvc -e "$pdf_previewer = 'sumatrapdf'" -e "$success_cmd = '""""""gvim"""""" --servername "GVIM" --remote-expr "vimtex#compiler#callback(1)"'" -e "$failure_cmd = '""""""gvim"""""" --servername "GVIM" --remote-expr "vimtex#compiler#callback(0)"'" "test.tex" >C:\Users\van\AppData\Local\Temp\VISE26E.tmp"
Error detected while processing function vimtex#compiler#compile[5]..1431[24]..1433[39]..1675[5]..1680:
line    6:
E484: Can't open file C:\Users\van\AppData\Local\Temp\VIoAE1C.tmp

The error popup there, but the file is compiled by latexmk correctly.

Finally, I try to add set noshelltemp as you suggested, the same problem is still here. My diff

$ git diff
diff --git a/autoload/vimtex/process.vim b/autoload/vimtex/process.vim
index a21b410..6798f43 100644
--- a/autoload/vimtex/process.vim
+++ b/autoload/vimtex/process.vim
@@ -100,7 +100,8 @@ endfunction
 " }}}1
 function! s:process._execute() abort dict " {{{1
   if self.silent
-    silent call system(self.prepared_cmd)
+       echo self.prepared_cmd
+       silent call system(self.prepared_cmd)
   elseif self.background
     silent execute '!' . self.prepared_cmd
     if !has('gui_running')
@@ -141,6 +142,7 @@ if has('win32')
             \]
       set shell& shellcmdflag& shellxquote& shellxescape&
       set shellquote& shellpipe& shellredir& shellslash&
+         set noshelltemp&
     else
       let self.win32_restore_shell = 0
     endif

If you find any unclear here, please ask me again.

[lervag]

Thanks, this is useful info.

Question: Does your fix (as below) solve this issue for you?

--- a/autoload/vimtex/process.vim
+++ b/autoload/vimtex/process.vim
@@ -100,6 +100,10 @@ endfunction
 " }}}1
 function! s:process._execute() abort dict " {{{1
   if self.silent
+    if self.output == 'null'
+      let self.prepared_cmd = '"' . self.prepared_cmd . '"'
+    endif
     silent call system(self.prepared_cmd)
   elseif self.background
     silent execute '!' . self.prepared_cmd

Follow-up question: If this does solve your issue, could you please check if everything else still works as expected (e.g. :VimtexClean)?

Now, the question about why the error message with the temp file appears. I have searched for some related issues, but I don't find anything that is surely related. There was one issue, justinmk/vim-gtfo#34; I'm not sure it is relevant. Do you use Git for windows with the cygwinshell thing?

[lervag]

This also looks related: google/vim-codefmt#86

What happens if you change this:

  function! s:process._prepare() abort dict " {{{1
    if &shell !~? 'cmd'
      let self.win32_restore_shell = 1
      let self.win32_saved_shell = [
            \ &shell,
            \ &shellcmdflag,
            \ &shellxquote,
            \ &shellxescape,
            \ &shellquote,
            \ &shellpipe,
            \ &shellredir,
            \ &shellslash
            \]
      set shell& shellcmdflag& shellxquote& shellxescape&
      set shellquote& shellpipe& shellredir& shellslash&
    else
      let self.win32_restore_shell = 0
    endif

to this:

  function! s:process._prepare() abort dict " {{{1
    let self.win32_restore_shell = 1
    let self.win32_saved_shell = [
          \ &shell,
          \ &shellcmdflag,
          \ &shellxquote,
          \ &shellxescape,
          \ &shellquote,
          \ &shellpipe,
          \ &shellredir,
          \ &shellslash
          \]
    set shell& shellcmdflag& shellxquote& shellxescape&
    set shellquote& shellpipe& shellredir& shellslash&

[lervag]

This also looks related: mattn/vim-gist#48

[vanabel]

1. Yes, my fix seems fixed the problem, and everything seems work fine with the fix on windows 7+gvim.

2. About justinmk/vim-gtfo#34
   I use gvim for windows on windows7 with PortableGit-2.19.1-64-bit.7z.exe, and add the PortableGit/bin path to my system path. I also have cygwin installed, but I don't think this is related to my issue (since I don't use vim/git through cygwin). As the PortableGit/bin has a bash.exe and a sh.exe, I also try to rename them to bash.bak.exe and sh.bak.exe, after I reload vimtex by :VimtexReload, the same error popup when I call :VimtexClean.

   On my gvim, the output of echo executable('cygpath') is 0 and echo (stridx(exepath('cygpath'), 'Git') is -1.

3. When I try to remove the if &shell !~? 'cmd' ... else ... endif, the same error popup when I call :VimtexClean.

[lervag]

Ok, thanks for the answers and for checking things!

I'll accept your suggested fix (with a minor modification). It might break something for someone else, in which case I might need to reverse it. But let's hope not!

[lervag]

I've pushed the change now. I'm sorry for not being able to answer your question in a more satisfiable manner, though. In any case, I hope things work better on Windows and I'm crossing my fingers that this does not break things for other windows users (e.g. Windows 10).

[lervag]

One thing: Do you get the same error if you try to run the prepared command in a windows cmd shell? I.e., run cmd, then copy the command, e.g. something like

start /b cmd /s /c "cd /D "C:\Users\van\Desktop" & latexmk -c  -outdir=C:\Users\van\vimfiles\myvim\vimtemp "test.tex" >nul"

Does this also give the same kind of error (I'm guessing it won't, but I'm curious what it will output).

---

Next: Can you try to run the same command inside Vim, but instead of running it with silent call system() (as in process.vim), you can run it with :silent execute '!' . cmd. I think this might actually work. So a different fix than the one you proposed might be to force Windows to use the !cmd syntax instead of system().

[vanabel]

I already run the command you ask on the cmd, which output

C:\Users\van>start /b cmd /s /c "cd /D "C:\Users\van\Desktop" & latexmk -c  -outdir=C:\Users\van\vimfiles\myvim\vimtemp "test.tex" >nul"

C:\Users\van>Latexmk: This is Latexmk, John Collins, 7 August 2018, version: 4.5
9. 

Then I must hit enter to stop this command, and no error popup.

---

I can run

:!start /b cmd /s /c "cd /D "C:\Users\van\Desktop" & latexmk -c  -outdir=C:\Users\van\vimfiles\myvim\vimtemp "test.tex" >nul" 

without any error. But I can't run

:silent exec "!cmd  /s /c "cd /D "C:\Users\van\Desktop" & latexmk -c  -outdir=C:\Users\van\vimfiles\myvim\vimtemp "test.tex" >nul" 

It says

E121: Undefined variable: cd
E15 Invalid expression: cd /D "C:\Users\van\Desktop" & latexmk -c -outdir=C:\Users\van\vimfiles\myvim\vimtemp "test.tex" >nul"

I think I do something wrong with the double quote.

[lervag]

Thanks. I think perhaps one way of solving the issue/issues could be to avoid using system() on Windows. If anyone should complain again that the recent commit breaks things on windows, then I'll try to fix again going that direction in stead.

But for now: Thanks for reporting and assisting in making vimtex better on Windows!