Startup time is slower for 1.3.0

[eatkins]

Due to the injected file management settings, sbt 1.3.0 generates roughly 50% more settings for a project compared to sbt 1.2.8. This causes project resolution to take longer. I randomly timed loading the https://github.com/softwaremill/sttp with 1.2.8 and 1.3.0-RC4 and it takes about 4 seconds to resolve the settings with 1.2.8 and about 6 seconds with 1.3.0. This manifested as total time for sbt exit to be roughly 13 seconds with 1.28 and 15 seconds with 1.3.0-RC4.

[dwijnand]

50% more default settings per project?? What does the load time change look like on a chunky multi-project build like Akka?

[eed3si9n]

I think we need to start incorporating benchmark into PR validation.
#3731

Not just for file management settings, but I've been blind to these regressions in my own changes that've introduced in the past as well.

[eatkins]

I don't think this is really that bad of a regression. On my computer akka takes about 20 seconds to load with 1.2.8 and 22.5 seconds with 1.3.0-RC4. That being said, it's relatively easy to eliminate most of the regression.

It seems that the number of settings rapidly grows in a multi project sbt build. The difference in sbt 1.3.0 vs 1.2.8 is that for every task that returned File, Seq[File], Path or Seq[Path], sbt injected 5 additional settings. There are about 20 settings in Keys.scala that are tasks returning File or Seq[File]. I guess these are defined in enough scopes that the number quickly grows. To reduce the number of injected settings, I stopped injecting them for File and Seq[File]. These settings are only there to support incremental task evaluation and weren't really serving any purpose for the legacy tasks. I think it's fine if users have to use the nio apis to use the incremental apis. I doubt there are any tasks returning Path or Seq[Path] in most builds since sbt has strongly pushed people towards java.io.File.

In akka, with 1.2.8, sbt resolved 38962 settings at startup and took 19.73 seconds to load. With 1.3.0-RC4, it resolved 62366 settings and loaded in 22.45 seconds. With 1.3.0x with File and Seq[File] injections removed, it drops to 47616 settings and loads in 20.27 seconds. At any rate, while setting resolution is a significant contributor to start up time, it isn't dominant. We're still only talking about 10-15% regression in startup time from 1.2.8 to 1.3.0-RC4 in spite of 60% more settings. We can get that down to a 2.5% regression in startup time with a two line change.

Long term, if we want to prevent startup time from increasing, we either have to rework Load to perform better, stop adding new keys, or only add new features via plugins.

Even in a CI environment where cold startup is unavoidable, 3 seconds is negligible when you consider the costs of starting up a vm and actually running the commands once sbt loads. Our fastest projects take something like 2 minutes to build on travis. It is also unlikely to affect many people's subjective experience of using sbt. On the command line, I personally can't really perceive much difference between 20 and 22 seconds load time. Either way I'm probably going to go look at something else while sbt loads. This is not to say that it doesn't matter, but that is the reason I said I didn't consider it high priority.

[dwijnand]

+2.5 seconds sounds fine. Thanks for the detail!

[eatkins]

I was able to get sbt 1.3.0 to load faster than 1.2.8 in spite of the extra settings in the akka project with #4981 and #4980.

[lbialy]

Wow, neat!

[eatkins]

Fixed by #4980 and #4981.