quickfix performance regression on Windows

[markonm]

Describe the bug
Interaction with quickfix window and quickfix list after patch 8.0.1781 suffered slowdown on Windows.

To Reproduce

" measureQuickFix.vim
let filename = tempname()
silent execute 'w' filename
let lines = map(repeat([filename], 10000), '{"filename": v:val}')

let start = reltime()
call setqflist(lines)
echom '  setqflist:' reltimestr(reltime(start))

let start = reltime()
silent copen
echom '  copen:    ' reltimestr(reltime(start))

let start = reltime()
call setqflist(lines)
echom '  setqflist:' reltimestr(reltime(start))

Run: gvim --clean -S measureQuickFix.vim

Output of above script:

v8.0.1780:

setqflist:   0.009004
copen:       0.009129
setqflist:   0.020638

v8.0.1781

setqflist:   0.009413
copen:       3.745755
setqflist:   3.794615

:copen and setqflist are much slower with 8.0.1781 included. Results are the same on the current master.

Expected behavior
Interaction with quickfix is fast as it was before patch 8.0.1781 was included.

Environment (please complete the following information):

• OS: Windows 10 2004

[vim-ml]

Hi,

On Sun, Sep 27, 2020 at 11:16 AM markonm ***@***.***> wrote: *Describe the bug* Interaction with quickfix window and quickfix list after patch 8.0.1781 <a796d46> suffered slowdown on Windows. *To Reproduce* " measureQuickFix.vim let filename = tempname() silent execute 'w' filename let lines = map(repeat([filename], 10000), '{"filename": v:val}') let start = reltime() call setqflist(lines) echom ' setqflist:' reltimestr(reltime(start)) let start = reltime() silent copen echom ' copen: ' reltimestr(reltime(start)) let start = reltime() call setqflist(lines) echom ' setqflist:' reltimestr(reltime(start)) Run: gvim --clean -S measureQuickFix.vim Output of above script: v8.0.1780: setqflist: 0.009004 copen: 0.009129 setqflist: 0.020638 v8.0.1781 setqflist: 0.009413 copen: 3.745755 setqflist: 3.794615 :copen and setqflist are much slower with 8.0.1781 included. Results are the same on the current master. *Expected behavior* Interaction with quickfix is fast as it was before patch 8.0.1781 was included. *Environment (please complete the following information):* - OS: Windows 10 2004

Can you try the attached patch? - Yegappan

[k-takata]

Can you try the attached patch?

The link is here: https://groups.google.com/d/msg/vim_dev/bau3jTheHMY/y2Z8d-StAQAJ

[markonm]

Thank you for the patch. I can see a big improvement in initial test.

v8.2.1767 with qfexpand.diff outputs:

setqflist:   0.011608
copen:       0.012365
setqflist:   0.019360

I also made a new test. This one has 46947 entries spread across 2307 files. The old one had 10000 entries and only 1 file.

" measureQuickFix2.vim
silent grep! /S "file" *
let qf = getqflist()

let start = reltime()
call setqflist(qf)
echom 'setqflist:' reltimestr(reltime(start))

let start = reltime()
silent copen
echom 'copen:    ' reltimestr(reltime(start))

let start = reltime()
call setqflist(qf)
echom 'setqflist:' reltimestr(reltime(start))

Steps used:

1. git clone --depth 1 https://github.com/vim/vim
2. cd vim
3. gvim --clean -S measureQuickFix2.vim

It produced the following output:

v8.0.1780

setqflist:   0.053862
copen:       0.031189
setqflist:   0.084630

v8.0.1781

setqflist:   0.052574
copen:      15.808420
setqflist:  16.696758

v8.2.1767

setqflist:   0.059002
copen:      16.761416
setqflist:  16.750146

v8.2.1767 with qfexpand.diff

setqflist:   0.058875
copen:       0.916748
setqflist:   0.946482

Although the patch greatly improves performance, interaction with quickfix is still a bit slower than in v8.0.1780.

[vim-ml]

Hi,

On Mon, Sep 28, 2020 at 9:03 AM markonm ***@***.***> wrote: Thank you for the patch. I can see a big improvement in initial test. v8.2.1767 with qfexpand.diff outputs: setqflist: 0.011608 copen: 0.012365 setqflist: 0.019360 I also made a new test. This one has 46947 entries spread across 2307 files. The old one had 10000 entries and only 1 file. " measureQuickFix2.vim silent grep! /S "file" * let qf = getqflist() let start = reltime() call setqflist(qf) echom 'setqflist:' reltimestr(reltime(start)) let start = reltime() silent copen echom 'copen: ' reltimestr(reltime(start)) let start = reltime() call setqflist(qf) echom 'setqflist:' reltimestr(reltime(start)) Steps used: 1. git clone --depth 1 https://github.com/vim/vim 2. cd vim 3. gvim --clean -S measureQuickFix2.vim It produced the following output: v8.0.1780 setqflist: 0.053862 copen: 0.031189 setqflist: 0.084630 v8.0.1781 setqflist: 0.052574 copen: 15.808420 setqflist: 16.696758 v8.2.1767 setqflist: 0.059002 copen: 16.761416 setqflist: 16.750146 v8.2.1767 with qfexpand.diff setqflist: 0.058875 copen: 0.916748 setqflist: 0.946482 Although the patch greatly improves performance, interaction with quickfix is still a bit slower than in v8.0.1780. The changes made by 1781 fixes the issue reported in

#2846 This change shortens the file name for each buffer. So the overhead of doing this will increase the time it takes to populate a quickfix list or to display the entries in the quickfix buffer (proportional to the number of unique buffers in the quickfix list). We can try to optimize the time it takes to shorten the buffer name. But we won't be able to get back to the same time as before this change. Regards, Yegappan

[vim-ml]

Yegappan wrote:

On Sun, Sep 27, 2020 at 11:16 AM markonm ***@***.***> wrote: > *Describe the bug* > Interaction with quickfix window and quickfix list after patch 8.0.1781 > <a796d46> > suffered slowdown on Windows. > > *To Reproduce* > > " measureQuickFix.vim > let filename = tempname() > silent execute 'w' filename > let lines = map(repeat([filename], 10000), '{"filename": v:val}') > > let start = reltime() > call setqflist(lines) > echom ' setqflist:' reltimestr(reltime(start)) > > let start = reltime() > silent copen > echom ' copen: ' reltimestr(reltime(start)) > > let start = reltime() > call setqflist(lines) > echom ' setqflist:' reltimestr(reltime(start)) > > Run: gvim --clean -S measureQuickFix.vim > > Output of above script: > > v8.0.1780: > > setqflist: 0.009004 > copen: 0.009129 > setqflist: 0.020638 > > v8.0.1781 > > setqflist: 0.009413 > copen: 3.745755 > setqflist: 3.794615 > > :copen and setqflist are much slower with 8.0.1781 included. Results are > the same on the current master. > > *Expected behavior* > Interaction with quickfix is fast as it was before patch 8.0.1781 was > included. > > *Environment (please complete the following information):* > > - OS: Windows 10 2004 Can you try the attached patch?

Since it's reported to be a nice improvement, can you make a pull request for this?

…

-- The Feynman problem solving Algorithm: 1) Write down the problem 2) Think real hard 3) Write down the answer /// Bram Moolenaar -- Bram@Moolenaar.net -- http://www.Moolenaar.net \\\ /// sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\ \\\ an exciting new programming language -- http://www.Zimbu.org /// \\\ help me help AIDS victims -- http://ICCF-Holland.org ///

[vim-ml]

Hi Bram,

On Mon, Sep 28, 2020 at 12:42 PM Bram Moolenaar ***@***.***> wrote: Yegappan wrote: > On Sun, Sep 27, 2020 at 11:16 AM markonm ***@***.***> wrote: > > > *Describe the bug* > > Interaction with quickfix window and quickfix list after patch 8.0.1781 > > < a796d46 > > > suffered slowdown on Windows. > > > > *To Reproduce* > > > > " measureQuickFix.vim > > let filename = tempname() > > silent execute 'w' filename > > let lines = map(repeat([filename], 10000), '{"filename": v:val}') > > > > let start = reltime() > > call setqflist(lines) > > echom ' setqflist:' reltimestr(reltime(start)) > > > > let start = reltime() > > silent copen > > echom ' copen: ' reltimestr(reltime(start)) > > > > let start = reltime() > > call setqflist(lines) > > echom ' setqflist:' reltimestr(reltime(start)) > > > > Run: gvim --clean -S measureQuickFix.vim > > > > Output of above script: > > > > v8.0.1780: > > > > setqflist: 0.009004 > > copen: 0.009129 > > setqflist: 0.020638 > > > > v8.0.1781 > > > > setqflist: 0.009413 > > copen: 3.745755 > > setqflist: 3.794615 > > > > :copen and setqflist are much slower with 8.0.1781 included. Results are > > the same on the current master. > > > > *Expected behavior* > > Interaction with quickfix is fast as it was before patch 8.0.1781 was > > included. > > > > *Environment (please complete the following information):* > > > > - OS: Windows 10 2004 > > Can you try the attached patch? Since it's reported to be a nice improvement, can you make a pull request for this?

I have created PR #7039 for this. #7039 Regards, Yegappan

[markonm]

To further eliminate the delay the following patch is needed. mch_isFullName is much slower on Windows because it calls mch_FullName.

diff --git a/src/os_mswin.c b/src/os_mswin.c
index 80009533c..429d93970 100644
--- a/src/os_mswin.c
+++ b/src/os_mswin.c
@@ -387,21 +387,10 @@ mch_FullName(
     int
 mch_isFullName(char_u *fname)
 {
-    // WinNT and later can use _MAX_PATH wide characters for a pathname, which
-    // means that the maximum pathname is _MAX_PATH * 3 bytes when 'enc' is
-    // UTF-8.
-    char szName[_MAX_PATH * 3 + 1];
-
     // A name like "d:/foo" and "//server/share" is absolute
-    if ((fname[0] && fname[1] == ':' && (fname[2] == '/' || fname[2] == '\\'))
-	    || (fname[0] == fname[1] && (fname[0] == '/' || fname[0] == '\\')))
-	return TRUE;
-
-    // A name that can't be made absolute probably isn't absolute.
-    if (mch_FullName(fname, (char_u *)szName, sizeof(szName) - 1, FALSE) == FAIL)
-	return FALSE;
-
-    return pathcmp((const char *)fname, (const char *)szName, -1) == 0;
+    return ((ASCII_ISALPHA(fname[0]) && fname[1] == ':'
+	    && (fname[2] == '/' || fname[2] == '\\'))
+	    || (fname[0] == fname[1] && (fname[0] == '/' || fname[0] == '\\')));
 }
 
 /*

Similar thing is also done in Neovim: https://github.com/neovim/neovim/blob/bfed67e00ecdf71e0c7d17b1fd802f223b42c800/src/nvim/path.c#L2294

Measured with measureQuickFix2.vim

v8.0.1780

setqflist:   0.053862
copen:       0.031189
setqflist:   0.084630

v8.0.2122

setqflist:   0.065174
copen:       0.907272
setqflist:   0.977449

v8.0.2122 with the patch

setqflist:   0.059616
copen:       0.038923
setqflist:   0.104571

Now the delay is almost completely gone.