Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Try to speed up dependency_hash() #655

Closed
wlandau opened this issue Jan 6, 2019 · 5 comments
Closed

Try to speed up dependency_hash() #655

wlandau opened this issue Jan 6, 2019 · 5 comments

Comments

@wlandau
Copy link
Collaborator

wlandau commented Jan 6, 2019

Ref: #647. Ideas:

  • Avoid calls to config$cache$exists().
  • Find an alternative to self_hash(). How do we reuse hashes without having to read them again from the cache?
@wlandau
Copy link
Collaborator Author

wlandau commented Jan 6, 2019

I think we can use in-memory memoization for self_hash(), as long as the memoized hashes do not carry over to the next make(). self_hash() seems to only be called inside dependency_hash(), so it is only used to get up-to-date hashes. If we can borrow some of those hashes across different targets, we will be in great shape.

@wlandau
Copy link
Collaborator Author

wlandau commented Jan 7, 2019

Avoiding calls to config$cache$exists() turned out to not do much. Memoization can help, but we need to be careful. We should only memoize hashes if the targets are already checked and up to date, and we want to forget everything we memoized in the next session of make(). Even after that, there is danger. I tried this, and some of the tests still failed. Will attempt again at some point, hopefully soon.

@wlandau
Copy link
Collaborator Author

wlandau commented Jan 8, 2019

@wlandau
Copy link
Collaborator Author

wlandau commented Jan 9, 2019

@wlandau
Copy link
Collaborator Author

wlandau commented Jan 9, 2019

Fixed via #660. Other bottlenecks are now more pressing.

@wlandau wlandau closed this as completed Jan 9, 2019
@wlandau wlandau added this to Done in Improve performance Jan 13, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
No open projects
Development

No branches or pull requests

1 participant