Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Already on GitHub? Sign in to your account

Cache projectile-project-root for performance #1149

Merged
merged 3 commits into from Jul 24, 2017

Conversation

Projects
None yet
6 participants
Contributor

alexmurray commented Jul 19, 2017

Cache the result of (projectile-project-root) as a buffer local variable to
increase performance - this is implemented the same as was done
for (projectile-project-name) in PR 1906.

Should help resolve issue #1003

Cache projectile-project-root for performance
Cache the result of (projectile-project-root) as a buffer local variable to
increase performance - this is implemented the same as was done
for (projectile-project-name) in PR 1906.

To ensure project root cache is used when there is no
project (ie. when (projectile-project-root) is nil) - we set the cached value
to the symbol 'none' and then replace this with nil before we return it - since
the cached value is non-nil we don't reevaluate it - but we don't want to
return 'none' as a result - so we instead substitute 'none' for nil before
returning, and also move the `projectile-require-project-root` error check to
the exit of the function to ensure this is still raised in this case.

Finally, this commit also changes the unit tests to handle the cached project
root by resetting before each evaluation - since the unit tests don't actually
change the (buffer-file-name) we need to do this to ensure they still pass.
Contributor

fommil commented Jul 22, 2017

I'd like to give this a try because I've been hit by perf in the past.

projectile.el
+ (progn
+ (setq projectile-cached-buffer-file-name buffer-file-name)
+ (setq projectile-cached-project-root
+ (or (let* ((dir default-directory)
@bbatsov

bbatsov Jul 22, 2017

Owner

You forgot the move the comment explaining this code here.

@alexmurray

alexmurray Jul 23, 2017

Contributor

Ah thanks.

@@ -911,10 +937,13 @@ This variable is reset automatically when Projectile detects that
the `buffer-file-name' has changed. It can also be reset manually
by calling `projectile-reset-cached-project-name'.")
-(defvar-local projectile-cached-buffer-file-name nil
@bbatsov

bbatsov Jul 22, 2017

Owner

Why did you delete this?

@alexmurray

alexmurray Jul 23, 2017

Contributor

I just moved it up earlier in the file so it is declared before we now first reference it.

@bbatsov

bbatsov Jul 24, 2017

Owner

No need for this. I can squash from the web ui.

projectile.el
- (if projectile-require-project-root
- (error "You're not in a project")
- default-directory))))
+ (or (cl-subst nil 'none
@bbatsov

bbatsov Jul 22, 2017

Owner

I don't understand what exactly is this supposed to do. Seems like some unnecessary complication of the code.

@alexmurray

alexmurray Jul 23, 2017

Contributor

To handle the case where there is no project root (and (projectile-project-root) returns nil) we can't just store nil as the cached value - as then we would end up reevaluating it every time and never using the cache. So instead we store the symbol 'none there instead - however we don't want to return 'none from the function since this invalidates the expected contract (return nil if not in a project) so we use cl-subst to replace any 'none value with nil.

@bbatsov

bbatsov Jul 23, 2017

Owner

Can't you simply use 'none as the default value for the buffer-local-variable?

@alexmurray

alexmurray Jul 24, 2017 edited

Contributor

But then we will never evaluate it - the code is:

(or (and (equal projectile-cached-buffer-file-name buffer-file-name)
          projectile-cached-project-root)
  ...)

So a non-nil value means the buffer-local cached value will be used - so it needs to be nil initially, then it gets evaluated once, and either set to a path or 'none and then it can be used as a cached value without reevaluation in the future.

@fommil

fommil Jul 24, 2017

Contributor

Isn't it easier if nil just cache misses? So what, projectile is slightly slower on directories that are not source managed or have a manual .projectile :-/

There is an elisp package for caching btw, if you'd like to reuse it.

@alexmurray

alexmurray Jul 24, 2017 edited

Contributor

I disagree - I often use emacs on directories which are not part of a project and having EVERYTHING slow down (say when using spaceline-all-the-icons which calls (projectile-project-root) on every redisplay) is very annoying. Why go to the effort of caching in a project and not also add support for caching outside of a project?

Also it is more expensive to determine when we are outside of a project than inside a project since we can't short-circuit out of the checks - we have to exhaust every possible project test each time - so it is MORE of a win performance wise to cache in this case (ie when no project) than in the case of being in a project.

Which elisp package are you referring to?

@bbatsov

bbatsov Jul 24, 2017

Owner

Just explain in a comment what this code is doing for future reference and I'll merge it. No need for more 3rd party deps.

@alexmurray

alexmurray Jul 24, 2017

Contributor

How about c473d17 ? Do you want me to squash these all into a single commit?

@bbatsov bbatsov merged commit 7951b17 into bbatsov:master Jul 24, 2017

1 check passed

continuous-integration/travis-ci/pr The Travis CI build passed
Details
Owner

bbatsov commented Jul 24, 2017

Thanks!

- "The last known value of `buffer-file-name' for the current
-buffer. This is used to detect a change in `buffer-file-name',
-which triggers a reset of `projectile-cached-project-name'.")
+(defun projectile-reset-cached-project-root ()
@aaronjensen

aaronjensen Jul 24, 2017

Would it make sense to call this during projectile-invalidate-cache?

Contributor

Silex commented Jul 24, 2017

I wonder if it still makes sense to cache projectile-cached-project-name

Contributor

fommil commented Jul 24, 2017

(in hindsight, I agree it makes sense to cache non-projects. I have worked in environments with network drives and ... short of having an extensive exclusion list for projectile... a cache for buffers in those folders would have surely helped)

JeanMax commented Aug 9, 2017

Hey!

This merge broke (projectile-all-project-files) for me.
It now only returns the files of the current project... so (projectile-find-file-in-known-projects) acts as (projectile-find-file); 'projectile-known-projects is set correctly.

I could reproduce the behaviour with 'emacs -q' and 2 random projects, but it seems that it doesn't affect everybody.
linux / emacs 25.2.1

@alexmurray alexmurray deleted the alexmurray:cache-project-root branch Aug 10, 2017

danielma added a commit to danielma/helm-projectile that referenced this pull request Aug 21, 2017

Separate buffers for separate projects
bbatsov/projectile#1149 introduced
buffer-local caching of certain project variables, which in turn
has caused issues with other commands not working properly. One
example is bbatsov/projectile#1151

When using `helm-projectile`, `helm-projectile-find-file` is broken
after switching buffers, because all helm commands are run in the
context of a global `helm-projectile` buffer, incorrect cache values
are return from functions like `(projectile-project-root)`

My particular issue has been that after switching to a new project, or
moving between windows, `helm-projectile-find-file` has opened helm
with a list of files from _the previous project._

I was able to fix this initially by adding
`(projectile-reset-cached-project-root)` into the
[`:candidates` for `helm-source-projectile-files-list`](https://github.com/bbatsov/helm-projectile/blob/0e9ba276b3fbc420b8cbdc1b99262ccd5c750db7/helm-projectile.el#L503)
but that would seem to break the point of the project caching that was
recently introduced.

By executing helm commands in a separate buffer for each project, we
are able to take advantage of performance boosts from buffer-local caches

@danielma danielma referenced this pull request in bbatsov/helm-projectile Aug 21, 2017

Open

Separate buffers for separate projects #81

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment