Performance observations (sbt 1.0 startup performance)

[jrudolph]

Here's an adhoc observation I've seen in a flamegraph I recently captured with async-profiler during startup of sbt. On runs where all .sbt files should already have been compiled, they still seem to be parsed which is where quite some time is spent because a compiler instance needs to be initialized for that (with all the classloading that happens because of that).

Another hotspot seems to be json deserialization of caches. Might make sense to optimize at least big caches like classpaths to use something better performing than json serialization.

flamegraph-akka-http-sbt.svg.txt

[jvican]

On runs where all .sbt files should already have been compiled, they still seem to be parsed which is where quite some time is spent because a compiler instance needs to be initialized for that (with all the classloading that happens because of that).

To the best of my knowledge this is the intended behaviour. Sbt caches at the statement level because it hoists up every statement to its own class -- I cannot think of a way we can remove that without changing the whole evaluation algorithm. You can check this in the .config-classes directory typically placed in project/target.

[jrudolph]

Superseded by #4012.

[jrudolph]

I cannot think of a way we can remove that without changing the whole evaluation algorithm

It was possible to do that by just introducing another layer of caching just for the result of splitExpressions. See jrudolph@b7a9f74.