[Fix #1147] Introduce a faster indexing method

The turbo-alien indexing method optimizes to the limit the speed
of the alien indexing method.  This means that Projectile will
not do any processing of the files returned by the external
commands and you're going to get the maximum performance
possible.  This behaviour makes a lot of sense for most people,
as they'd typically be putting ignores in their VCS config and
won't care about any additional ignores/unignores/sorting that
Projectile might also provide.

CHANGELOG.md

@@ -8,6 +8,7 @@
* New interactive command `projectile-run-ielm`.
* Add [crystal](https://crystal-lang.org) project type.
* [#850](https://github.com/bbatsov/projectile/issues/850): Make it possible to prompt for a project, when you're not in a project, instead of raising an error. (see `projectile-require-project-root`).
* [#1147](https://github.com/bbatsov/projectile/issues/1147): Introduce a faster indexing method - `turbo-alien`.

### Changes

doc/configuration.md

@@ -6,17 +6,25 @@ want to fine-tune to make Projectile fit your workflow better.

## Project indexing method

Projectile has two modes of operation - one is portable and is
implemented in Emacs Lisp (therefore it's *native* to Emacs and is known
as the `native indexing method`) and the other relies on external
commands like `find`, `git`, etc to obtain the list of files in a
project.

Projectile has three modes of operation - one is portable and is
implemented in Emacs Lisp (therefore it's *native* to Emacs and is
known as the `native indexing method`) and the other two (`alien` and
`turbo-alien`) rely on external commands like `find`, `git`, etc to
obtain the list of files in a project.

The `turbo-alien` indexing method optimizes to the limit the speed of
the `alien` indexing method.  This means that Projectile will not do
any processing of the files returned by the external commands and
you're going to get the maximum performance possible.  This behaviour
makes a lot of sense for most people, as they'd typically be putting
ignores in their VCS config (e.g. `.gitignore`) and won't care about
any additional ignores/unignores/sorting that Projectile might also
provide.

!!! Info

    Since the `native` indexing mode is much slower, by default the `alien`
    method is used on all operating systems except Windows.
    By default the `turbo-alien` method is used on all operating systems except Windows.
    Prior to Projectile 1.1 `alien` used to be the default.

To force the
use of native indexing in all operating systems:
@@ -25,16 +33,23 @@ use of native indexing in all operating systems:
(setq projectile-indexing-method 'native)
```

To force the use of alien indexing in all operating system:
To force the use of alien indexing in all operating systems:

```el
(setq projectile-indexing-method 'alien)
```

This can speed up Projectile in Windows significantly. The
disadvantage of this method is that it's not well supported on Windows
systems, as it requires setting up some Unix utilities there. If
there's problem, you can always use native indexing mode.
To force the use of turbo alien indexing in all operating systems:

```el
(setq projectile-indexing-method 'turbo-alien)
```

This can speed up Projectile in Windows significantly (especially on
big projects). The disadvantage of this method is that it's not well
supported on Windows systems, as it requires setting up some Unix
utilities there. If there's problem, you can always use `native`
indexing mode.

## Caching

projectile.el

@@ -78,7 +78,7 @@
  :link '(url-link :tag "Online Manual" "https://docs.projectile.mx/")
  :link '(emacs-commentary-link :tag "Commentary" "projectile"))

(defcustom projectile-indexing-method (if (eq system-type 'windows-nt) 'native 'alien)
(defcustom projectile-indexing-method (if (eq system-type 'windows-nt) 'native 'turbo-alien)
  "Specifies the indexing method used by Projectile.

There are two indexing methods - native and alien.
@@ -93,12 +93,22 @@ The alien indexing method uses external tools (e.g. git, find,
etc) to speed up the indexing process.  The disadvantage of this
method is that it's not well supported on Windows systems.

By default alien indexing is the default on all operating
The turbo-alien indexing method optimizes to the limit the speed
of the alien indexing method.  This means that Projectile will
not do any processing of the files returned by the external
commands and you're going to get the maximum performance
possible.  This behaviour makes a lot of sense for most people,
as they'd typically be putting ignores in their VCS config and
won't care about any additional ignores/unignores/sorting that
Projectile might also provide.

By default turbo-alien indexing is the default on all operating
systems, except Windows."
  :group 'projectile
  :type '(radio
          (const :tag "Native" native)
          (const :tag "Alien" alien)))
          (const :tag "Alien" alien)
          (const :tag "Turbo Alien" turbo-alien)))

(defcustom projectile-enable-caching (eq projectile-indexing-method 'native)
  "When t enables project files caching.
@@ -1057,10 +1067,12 @@ Files are returned as relative paths to the project ROOT."
                         (gethash directory projectile-projects-cache))))
    ;; cache disabled or cache miss
    (or files-list
        (if (eq projectile-indexing-method 'native)
            (projectile-dir-files-native root directory)
        (pcase projectile-indexing-method
          ('native (projectile-dir-files-native root directory))
          ;; use external tools to get the project files
          (projectile-adjust-files (projectile-dir-files-external directory))))))
          ('alien (projectile-adjust-files (projectile-dir-files-external directory)))
          ('turbo-alien (projectile-dir-files-external directory))
          (_ (user-error "Unsupported indexing method `%S'" projectile-indexing-method))))))

(defun projectile-dir-files-native (root directory)
  "Get the files for ROOT under DIRECTORY using just Emacs Lisp."