10x longer startup times when using specific settings

[ThSGM]

Description

I am trying to diagnose what can be done to reduce my vimtex startup times. It is quite significant at the moment. I have prepared a video below using this profiler. Basically, when I comment in the following code:

let g:vimtex_view_method = "skim"
let g:vimtex_compiler_progname = "nvr"
let g:vimtex_view_general_viewer
	\ = '/Applications/Skim.app/Contents/SharedSupport/displayline'
let g:vimtex_view_general_options = '-r @line @pdf @tex'

the neovim startup time is about 10x longer than if I comment out that code. I have attached the profile.log here.

ScreenFlow.mp4

I can try and prepare a minimal test file, but that might require some more time for me to strip out the necessary stuff and re-run the tests. Perhaps this may do to get the ball rolling.

Steps to reproduce

No response

Expected behavior

No response

Actual behavior

No response

Do you use a latexmkrc file?

No

VimtexInfo

System info
  OS: Mac OS X 10.15.7 (19H1217)
  Vim version: NVIM v0.6.0-dev+59-g2475161de
  Has clientserver: true
  Servername: /var/folders/yr/htyy1zjx2pb9_1s11ns1yz1m0000gn/T/nvimB0St0M/0

VimTeX project: test
  base: test.tex
  root: /Users/myuser/Downloads/vim-plugins-profile-master
  tex: /Users/myuser/Downloads/vim-plugins-profile-master/test.tex
  main parser: current file
  document class: 
  compiler: latexmk
    engine: -pdf
    options:
      -verbose
      -file-line-error
      -synctex=1
      -interaction=nonstopmode
    callback: 1
    continuous: 1
    executable: latexmk
  viewer: Skim
  qf method: LaTeX logfile

[lervag]

First: You don't need to set both g:vimtex_view_method = 'skim' and g:vimtex_view_general_.... The latter controls the viewer if you use the general/generic method. You don't. So just remove those two lines. You can also remove the g:vimtex_compiler_progname line, because it should be automatic. I.e., use the following, much simpler config:

let g:vimtex_view_method = "skim"

However, this should not be relevant to your issue. Can you confirm that you still reproduce your problem?

For profiling, I suggest to just use the built-in stuff. The startup-profiling is very lacking in detail, so it does not explain why something is slow.

So, can you try this:

" test.vim
set nocompatible
let &rtp = '~/.local/share/nvim/plugged/vimtex,' . &rtp
let &rtp .= ',~/.local/share/nvim/plugged/vimtex/after'
filetype plugin indent on
syntax enable

call vimtex#profile#file('test.tex')
quitall!

With some suitable test.tex file, run nvim --headless -u test.vim. It should generate a prof.log file. The bottom part of this profile log shows the totals. You could just upload the file in the issue thread if you want me to look at it.

[ThSGM]

This took me a few days to do.

I ran both your instructions and also using nvim test.tex --startuptime start.log. I don't understand why but the second way seemed to be slower (just my interpretation...maybe it was the headless mode).

Here is the output of prof.log.

Here is the output of start.log

[clason]

That seems like a perfectly normal startup time to me (adjusting for performance differences of machines). In particular, skim.vim is slow presumably because it does file system access through the osa scripting host -- maybe there is a better way to check for Skim?

You could try commenting out

vimtex/autoload/vimtex/view/skim.vim

Lines 15 to 18 in e6c03a1

	if system(l:cmd)
	call vimtex#log#error('Skim is not installed!')
	return {}
	endif

to see if that improves load times.

The other big contribution comes from vim-pencil -- so this is not VimTeX's fault.

[clason]

For example, VimTeX could just check whether

osascript -e 'tell application "Finder" to get id of application "Skim"'

returned net.sourceforge.skim-app.skim; that is more than twice as fast (70ms vs 170ms) for me.

Alternatively, don't test for Skim on startup at all and just do the (full) check on :VimtexInfo? The way app installation works on macOS, I have a hard time imagining that a user would need to have to rely on this check to know whether Skim is installed or not; in any case, that might be something to cache, too, as doing that on every startup seems a bit wasteful to me?

[clason]

An alternative would be to gate the check behind a g:vimtex_viewer_check variable (default true) that "advanced" users can set to false to skip checks they know are not needed?

[clason]

That works, too ;) Can confirm that this speeds up the startup noticeably!

[lervag]

I've pushed a minor update that delays the check for wether Skim is available. I think the minor delay should be more or less ignorable in this context, and so there should be no reason to cache or add an option.

Let me know what you think.

Btw., do I understand correctly @clason, that I can make the following change to improve the speed further? (Would be good if you tested this on your side before I push it.)

    " Old
    let l:cmd = join([
          \ 'osascript -e ',
          \ '''tell application "Finder" to POSIX path of ',
          \ '(get application file id (id of application "Skim") as alias)''',
          \])
    let self._skim_available = !system(l:cmd)

    " New
    let l:cmd = 'osascript -e '
          \ . '''tell application "Finder" to get id of application "Skim"'''
    let self._skim_available = !system(l:cmd)

[clason]

But right now I get an error when opening the viewer:

Error detected while processing function vimtex#view#view[2]..22[1]..21:
line   11:
E716: Key not present in Dictionary: "self._skim_available"

(without doing anything else first)?

[lervag]

Sorry, minor error on my side. Fixed now.

[clason]

Btw., do I understand correctly @clason, that I can make the following change to improve the speed further?

No, not really; that command outputs in any case; if Skim is not installed, you get

33:35: execution error: Can’t get application "Skim". (-1728)

(from the command line).

[lervag]

Ok. If you think the optimization is interesting, then feel free to propose a change. I assume it would also require a change to the let self._skim_available = !system() line to instead consider the actual output. But since I can't test this myself it seems risky to just push this.

[clason]

Yes, you need to check whether the return string is equal to net.sourceforge.skim-app^@ -- not sure how to test this with the weird special characters... Not sure if it's worth it for 100ms?

[lervag]

Can you test this:

  let l:output = get(split(, '\n'), 0, '')
  let self._skim_available = l:output ==# 'net.sourceforge.skim-app'

  " or
  let self._skim_available =
        \ match(system(l:cmd), '^net.sourceforge.skim-app') >= 0

[clason]

Yes, the latter works:

diff --git a/autoload/vimtex/view/skim.vim b/autoload/vimtex/view/skim.vim
index 15e8996..e951790 100644
--- a/autoload/vimtex/view/skim.vim
+++ b/autoload/vimtex/view/skim.vim
@@ -52,12 +52,10 @@ function! s:skim.skim_available() abort " {{{1
     let self._requirements_checked = v:true
 
     " Check if Skim is installed
-    let l:cmd = join([
-          \ 'osascript -e ',
-          \ '''tell application "Finder" to POSIX path of ',
-          \ '(get application file id (id of application "Skim") as alias)''',
-          \])
-    let self._skim_available = !system(l:cmd)
+    let l:cmd = 'osascript -e '
+          \ . '''tell application "Finder" to get id of application "Skim"'''
+    let self._skim_available =
+          \ match(system(l:cmd), '^net.sourceforge.skim-app') >= 0
     if !self._skim_available
       call vimtex#log#error('Skim is not installed!')
     endif

(Don't know if it's faster due to the string matching, but I would assume so.)

[lervag]

Yes, I believe the string matching is O(1 ms), so that should not really matter much. I've updated now, please test.

@ThSGM I'm curious to hear if the latest commits improves your experience.

[clason]

Looking good to me!

[lervag]

Alternatively, don't test for Skim on startup at all and just do the (full) check on :VimtexInfo? The way app installation works on macOS, I have a hard time imagining that a user would need to have to rely on this check to know whether Skim is installed or not; in any case, that might be something to cache, too, as doing that on every startup seems a bit wasteful to me?

I've now pushed a relatively large update where I've implemented a cached version of vimtex#jobs#capture, i.e. vimtex#jobs#cached, which does essentially what you suggest here. That is, the _skim_available system call is now cached and after the initial run it should be more or less instant.

@ThSGM I'm closing this issue now, as I believe there has been multiple improvements. It would be nice to hear back from you - do the updates resolve your issue?

[ThSGM]

@ThSGM I'm closing this issue now, as I believe there has been multiple improvements. It would be nice to hear back from you - do the updates resolve your issue?

Hi @lervag,

My apologies, since this will be a non-update from me, but I felt it should be important for me to at least temporarily provide a reply to inspire others to look into these issues.

Unfortunately, our academic term has resumed in full force, and I had to temporarily give up using neovim + vimtex to write my LaTeX. I put in a good 8-12 months into the endeavor, but in the end, it was too difficult to sort out the slowness issues. It was unclear if this was an issue with excessive plugins, combined with natural vimtex limitations, and large LaTeX documents (a book with multiple chapters, for instance). Trying to diagnose slowdown was like playing whack-a-mole.

The simple truth is that Sublime Text + LaTeXTools currently gives me a much faster and unproblematic workflow. It also has significant advantages in fuzzy finding references and citations. And it is near-instantaneous in comparison to the slow mess I had with vim.

I'd like to return to vimtex in the future since I do want to convert to a cross-platform vim setup. However, at the moment, this too severely impacts my work.

Thanks for your help. I am on the various subreddits and am subscribed to watch to vimtex development and I'll keep an eye. I hope that the speed issues can be improved in the future, particularly when used in conjunction with macOS.

[lervag]

My apologies, since this will be a non-update from me, but I felt it should be important for me to at least temporarily provide a reply to inspire others to look into these issues.

No problem, no need to apologize. This is life, it is complicated, and we have to make pragmatic choices! :)

The simple truth is that Sublime Text + LaTeXTools currently gives me a much faster and unproblematic workflow. It also has significant advantages in fuzzy finding references and citations. And it is near-instantaneous in comparison to the slow mess I had with vim.

I'm glad to hear that Sublime++ works well for you. For me, neovim with VimTeX is fast and works more or less exactly as I want it to work, and I try to help make it work as best as possible for all users.

I'd like to return to vimtex in the future since I do want to convert to a cross-platform vim setup. However, at the moment, this too severely impacts my work.

Thanks for your help. I am on the various subreddits and am subscribed to watch to vimtex development and I'll keep an eye. I hope that the speed issues can be improved in the future, particularly when used in conjunction with macOS.

You're of course more than welcome back at any time. I'll not be able to improve speed issues unless they are reported well and I'm able to reproduce or at the very least understand the main causes. But I'll continue my efforts, and with time and help from users I hope to improve things further in the future.