Repository navigation
[2.x] Reduce number of long-living instances to speed up loading - #7866
Merged
Merged
Conversation
Member
|
@adpi2 Nice work! |
eed3si9n
reviewed
Nov 4, 2024
eed3si9n
reviewed
Nov 4, 2024
eed3si9n
approved these changes
Nov 5, 2024
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
After loading, instances of
ScopedKey,Scope,SelectorProjectRefrepresent 25.6% of the total number of instances, and 27.3% of the heap size. The main idea of this PR is to reduce the number of duplicated instances using uniqueness caches.Uniqueness cache on
ScopeIt reduces the number of
Scopeinstances by 94%. On its own, it slows down the loading, becauseScope.applybecomes more expensive and it is called too many times. This can be compensated by using temporary caches inProject.transformanddelegates: Scope => Seq[Scope], to avoid callingScope.applyto create too many short-lived instances.It also reduces the number of
Selectby 92% andProjectRefby 99.8%, which make them totally negligible.Since we use a uniqueness cache we might as well pre-compute the hash code of each scope, which reduce the CPU time.
Uniqueness cache on
ScopedKey(uncommited)I think it is also worth mentioning that I tried a uniqueness cache on
ScopedKeybut it was not really better. Indeed, on one hand it reduced the number ofScopedKeyby 59.6%, but on the other hand it was counterbalanced by the number of nodes in the cache itself.Other changes
The first commit reduces the number of
Init.GetValueby 48% for free.Result
I am still using sttp to collect those metrics, on my 12 CPUs laptop, and JDK temurin 17 (G1GC).
It shows up a 20% speedup of the loading time compared to sbt 2.0.0-M2, and a 41% speedup compared to sbt 1.10.2. Those are rough estimations.
Compared to 2.0.0-M2
Compare to 1.10.2