Performance

[quicknir]

I'm working with a project of some 30K files. projectile-use-cache is t. With helm, find file starts almost instantly. With counsel, it takes like 2-3 seconds. Is it expected that the performance is that much worse?

[ericdanan]

I don't think ivy should be slower than helm in general, so probably counsel-projectile does something less efficiently than helm-projectile. I don't have such a large project to test, could you point me to one perhaps on github that I could clone?

Please beware that I am really out of time at the moment so this will likely take me a while, sorry about that.

[ericdanan]

Could you let me know if the issue is still there when calling counsel-projectile-find-file instead of counsel-projectile?

Also, what happens if you use projectile with ivy as completion system but not counsel-projectile? That is, evaluate

(counsel-projectile-off)
(setq projectile-completion-system 'ivy)

then call projectile-find-file.

[freesteph]

@ericdanan I'm encountering the same issue, followed your instructions in my scratch buffer, and everything is much faster now. Can you let me know what else I could provide to help us get to the bottom of this?

[unhammer]

I also noticed that this was a lot slower than just running projectile with ivy. Profiler gives:

- command-execute                                                6813  88%
 - call-interactively                                            6813  88%
  - funcall-interactively                                        6665  86%
   - counsel-projectile                                          6658  86%
    - if                                                         6658  86%
     - ivy-read                                                  6658  86%
      - counsel-projectile--global-list                          6653  86%
       - append                                                  6653  86%
        - mapc                                                   6653  86%
         - counsel-projectile--buffer-list                       6653  86%
          - ivy--buffer-list                                     6653  86%
           - all-completions                                     6653  86%
            - internal-complete-buffer                           6653  86%
             - #<lambda 0x6cfc371>                               6653  86%
              - member                                           6653  86%
               - projectile-project-buffer-names                 6653  86%
                - -map                                           6653  86%
                 - projectile-project-buffers                    6653  86%
                  - let*                                         6653  86%
                   - -filter                                     6569  85%
                    - #<lambda 0x3ce5a75b9a87aa6b>               6537  84%
                     - projectile-project-buffer-p               6509  84%
                      - save-current-buffer                      5331  69%
                       - and                                     4380  56%
                        - string-prefix-p                        3743  48%
                         - file-truename                         3667  47%
                          - file-truename                        2944  38%
                           - file-truename                       2349  30%
                            - file-truename                      1727  22%
                             - file-truename                     1128  14%
                              - file-truename                     677   8%
                               - file-truename                    384   4%
                                - tramp-completion-file-name-handler                176   2%
                                 - tramp-completion-run-real-handler                176   2%
                                  - apply                         176   2%
                                   - file-truename                168   2%
                                    + file-symlink-p                124   1%
                                    + file-name-nondirectory                 16   0%
                                    + directory-file-name                 12   0%
                                    + file-name-directory                  4   0%
                                + file-truename                   140   1%
                                + file-name-as-directory                  4   0%
                               + tramp-completion-file-name-handler                129   1%
                              + tramp-completion-file-name-handler                268   3%
                              + file-name-as-directory                 12   0%
                             + tramp-completion-file-name-handler                132   1%
                             + file-symlink-p                     114   1%
                             + file-name-as-directory                 25   0%
                            + tramp-completion-file-name-handler                159   2%
                            + file-name-as-directory                  8   0%
                           + debug                                 83   1%
                             file-name-as-directory                  4   0%
                           eq                                       4   0%
                        + not                                     457   5%
                        + debug                                   100   1%
                        + string-equal                             76   0%
                      buffer-list                                   4   0%
                   + projectile-project-root                       44   0%
      + projectile-prepend-project-name                             5   0%
   + execute-extended-command                                       7   0%
  + byte-code                                                     148   1%
+ ...                                                             791  10%
+ timer-event-handler                                              83   1%
+ redisplay_internal (C function)                                   8   0%
  global-undo-tree-mode-check-buffers                               6   0%
+ evil-normal-post-command                                          6   0%
+ projectile-global-mode-check-buffers                              4   0%

[ericdanan]

See #44

[quicknir]

@ericdanan Just as a sort of heads up, some of my investigations into projectile more recently showed that in addition to additional lag from counsel, projectile itself is also ridiculously slow. On a project of 30K files, without caching, projectile (whether with helm or with counsel) takes 10+ seconds. But a simple find command, or even faster, ag -g, takes < 2. Even with caching, the responsiveness of projectile is just terrible. On the other hand, a but of custom code that I whipped up myself (leveraging a lot of helm ag), is far more responsive and doesn't require a cache.

When I looked into the code I remember vaguely seeing that proejctile does caching in an insane way. It actually stores all of the filenames as a variable in a lisp file (!!!) and then you read the cache by reading this entire file basically. The approach is kind of crazy compared to just having a text file with one filename per line that would be dead easy and fast to search.

Anyway my plan is just to move away from projectile as fast as possible at this point (which sadly, may not be that fast). Thanks for your effort and help, cheers.