Git Tab Completion Slow on Larger repositories for certain commands

[nwhansen]

• I was not able to find an open or closed issue matching what I'm seeing

Setup

• Which version of Git for Windows are you using? Is it 32-bit or 64-bit?

64 bit installer
git version 2.16.1.windows.4
cpu: x86_64
built from commit: ef6d451bbfef86a529ebf12620289e0f15a93d8e
sizeof-long: 4

• Which version of Windows are you running? Vista, 7, 8, 10? Is it 32-bit or 64-bit?
  Windows 10

Microsoft Windows [Version 10.0.15063]
(c) 2017 Microsoft Corporation. All rights reserved.

• What options did you set as part of the installation? Or did you choose the
  defaults?

Editor Option: VIM
Path Option: Cmd
SSH Option: OpenSSH
CURL Option: OpenSSL
CRLF Option: CRLFCommitAsIs
Bash Terminal Option: MinTTY
Performance Tweaks FSCache: Enabled
Use Credential Manager: Enabled
Enable Symlinks: Disabled

• Any other interesting things about your environment that might be related
  to the issue you're seeing?

This repository is very large. 5gb in size however this is replicatable with linux kernal repository. I am not using a solid state for the repository.

Details

• Which terminal/shell are you running Git from? e.g Bash/CMD/PowerShell/other

Git Bash

• What commands did you run to trigger this issue? If you can provide a
  Minimal, Complete, and Verifiable example
  this will help us understand the issue.

git status -- ar[TAB]

Or

git mv ar[TAB]

• What did you expect to occur after running these commands?
  Tab completed to

git status -- arch/

• What actually happened instead?
  After approximately 10-20 seconds it completes

• If the problem was occurring with a specific repository, can you provide the
  URL to that repository to help us with testing?
  Reproduced using: https://github.com/torvalds/linux

---

h3. Additional investigation:
Enabling

GIT_TRACE_PERFORMANCE=<logfile>

Creates the following log entries

13:59:59.608590 trace.c:417             performance: 0.002119112 s: git command: 'C:\Program Files\Git\mingw64\bin\git.exe' 'rev-parse' '--git-dir' '--is-inside-git-dir' '--is-bare-repository' '--is-inside-work-tree' '--short' 'HEAD'
14:00:16.755165 trace.c:417             performance: 0.001872594 s: git command: 'C:\Program Files\Git\mingw64\bin\git.exe' 'config' 'status.showUntrackedFiles'
14:00:28.706112 trace.c:417             performance: 11.796146578 s: git command: 'C:\Program Files\Git\mingw64\bin\git.exe' '-C' '.' 'ls-files' '--exclude-standard' '--cached' '--directory' '--no-empty-directory' '--others'

Running the command

"C:\Program Files\Git\mingw64\bin\git.exe" -C ./ ls-files --exclude-standard --cached --directory --no-empty-directory --others

Returns 62913 lines. Attached as ls-files-result.txt
ls-files-result.txt

Git status on freshly checkout respository is the file checkoutStatus.txt
checkoutStatus.txt

Some further investigation shows that running set -x to gain some insight into the completion script shows that we spend a lot of time opening the completion script with

++case "$file" in
++echo drivers
++read -r file

being repeated what seems thousands of times.

[drizzd]

I can confirm your observation on Windows 10 64-bit with Msys2 mingw64 git version 2.15.1.windows.2:

~/src/linux$ COMP_WORDS=(git status -- ar) COMP_CWORD=3; time _git

real    0m11.876s
user    0m4.685s
sys     0m6.808s

Most of the time is spent in the while read loop in __git_index_files. We can improve much by replacing the loop with an sed script:

~/src/linux$ COMP_WORDS=(git status -- ar) COMP_CWORD=3; time
_git

real    0m1.372s
user    0m0.263s
sys     0m0.167s

Here's the change:

diff --git a/.git-completion.bash b/.git-completion-mod.bash
index a54cc2c..f20945f 100644
--- a/.git-completion.bash
+++ b/.git-completion-mod.bash
@@ -351,12 +351,7 @@ __git_index_files ()
        local root="${2-.}" file

        __git_ls_files_helper "$root" "$1" |
-       while read -r file; do
-               case "$file" in
-               ?*/*) echo "${file%%/*}" ;;
-               *) echo "$file" ;;
-               esac
-       done | sort | uniq
+       sed -e '/^\//! s#/.*##' | sort | uniq
 }

 # Lists branches from the local repository.

I cannot reproduce the issue in WSL. Maybe the mingw64 version of bash has a performance issue with while read loops.

[drizzd]

I have prepared a branch with the changes here https://github.com/drizzd/git/tree/completion-optimize-ls-files-filter. Still need to setup an email client to contribute this to the git mailing list.

[dscho]

@drizzd if you want, I can contribute this for you. Or you can follow @derrickstolee's excellent guide and set up git send-email: https://github.com/git-for-windows/git/blob/master/CONTRIBUTING.md#submit-your-patch

[dscho]

Maybe the mingw64 version of bash has a performance issue with while read loops.

FWIW this is very likely...

[nwhansen]

I modified the git completion script to include that change and the performance difference is quite significant. I will continue to test out the change and and report further findings.

[drizzd]

Thanks. Please note that the patch has been discussed on the mailing list: https://public-inbox.org/git/20180318012618.32691-1-szeder.dev@gmail.com/

The patch removes a sorting operation, which is wrong. I have a prepared a patch with the fix here:
https://github.com/drizzd/git/commits/completion-optimize-ls-files-filter4.

[dscho]

@drizzd do you think the updated patch is ready to be cherry-picked into Git for Windows?

[nwhansen]

My apologies I am not currently following that mailing list. I appreciate the effort and time looking into this bug report :)

[dscho]

My apologies I am not currently following that mailing list.

Only start following it if you like drinking from a fire hose...

[drizzd]

@dscho It's working fine for me on Windows. But who knows if there are more comments on the list. I can make a note for you here when the patch is accepted.

[dscho]

A new snapshot is building right now and should become available in about half an hour: https://wingit.blob.core.windows.net/files/index.html

[dscho]

(Please test, the easiest way is to install the portable Git, start its git-bash.exe and verify that the tab completion is faster in your repository.)

[dscho]

Please test

Can y'all please test?

[nwhansen]

It is working so far. The performance difference is outstanding, the native commands are faster but it is much harder to tell the difference. and they are well within a second of each other.

The double tab list is in the same order so I don't know if there is anything else I can really test. Thanks so much for getting a fix for this @drizzd @dscho

[dscho]

The performance difference is outstanding

That is really what I was hoping.

The double tab list is in the same order

Very good. Thank you for verifying, this is really crucial information to have confidence in this improvement.