counsel-git-grep slow to respond

[sooheon]

It takes ~3 seconds for the prompt to appear after calling it in a small repo. counsel-git is also slow to appear, though a little better. After the prompt appears the typing and narrowing is instantaneous, so something about just the initialisation is slow. counsel-ag is the opposite, the prompt appears immediately but narrowing results is slow.

How can I help track down what causes this delay?

[abo-abo]

I'm pretty sure this is the initial start-up cost of the git-grep shell tool provided by Git. It needs to index the repository once and set up, so that the future calls to git-grep are fast.

Just checked this for a clone of scipy that I haven't touched in a long while. First shell call to git grep asdfasdf lasts 2 seconds, next one is really fast:

$ time git grep asdfasdf

real    0m0.032s
user    0m0.036s
sys     0m0.064s

And of course ag works differently that git-grep and probably doesn't need an initial index step. Closing this for now, feel free to reopen if there's more evidence that Elisp is to blame for the delay.

[sooheon]

I'm getting the same nearly 0 time for git grep itself called in shell, but even on repeated calls counsel-git-grep is slow to start up.

[abo-abo]

Try evaluating this code in different repositories:

(benchmark-run 1
  (let ((counsel--git-grep-dir default-directory))
    (counsel--gg-count "" t)))

It's called once in each counsel-git-grep. In the Linux source it runs for 0.4s for me. The code itself calculates the number of lines in the repository, for the case of Linux it's 21M, so the time should be shorter for smaller repos.

[sooheon]

Even in my .emacs.d it's 1.931969s. Other random repos, non large, are ~1.5 and 2 seconds.

[abo-abo]

I see. So it's a big issue on slower computers. Perhaps some caching can be introduced. Or a faster way of determining the repository size. Suggestions?

[sooheon]

No real suggestions yet, some more info:

It's not really a slow computer, it's mid 2014 2.2Ghz i7.

GNU Emacs 25.0.94.1 (x86_64-apple-darwin15.4.0, NS appkit-1404.46 Version 10.11.4 (Build 15E65)) of 2016-05-27

Git 2.8.3

[abo-abo]

Here's the command used to calculate the number of lines in a repository:

git --no-pager grep -c -n --no-color -i -e "" | sed 's/.*:\(.*\)/\1/g' | awk '{s+=$1} END {print s}'

See if you can optimize it for your system.

[sooheon]

It seems like something else is causing the slowdown, because that command runs pretty much instantaneously in the terminal for me (0.05s).

[sooheon]

Now that I've had some sleep I realise it's obviously because of the startup time of the shell itself. shell-file-name had been set to fish, and fish is much slower to start than bash is.

[abo-abo]

lol

[edkolev]

I have a somewhat big repo and somewhat limited IO bandwidth, as a result counsel--gg-count takes ~5 seconds on a cold repo. I experimented with different ways to count the total number of lines in a git repo, but couldn't find a significantly faster approach.

@abo-abo, would you be willing to accept a PR which adds an option to skip calling counsel--gg-count in counsel-git-grep, so the repo will always be treated as big (> 20000 lines) and use ivy's :dynamic-collection?

[abo-abo]

@edkolev sounds good, thanks in advance.