Long loading time on multi-project

[huajiang-tubi]

We got ~140 more seconds on sbt reload when sbt-version-policy is added to our multi-project with 10 libraries under the root. The problem turned out to be that, the versionPolicyPreviousVersions setting is resolved at load time by searching the resolvers for previous versions of the project artifacts. It take ~14 seconds for each subproject. Typically we work on a single project at a time, that makes fetching previous versions for all the other projects a waste.

A potential optimization I'm thinking about is to make versionPolicyPreviousVersions a taskKey instead of settingKey, so the versions will be fetched only when actually needed. It would also need to make settingKeys depending on it taskKeys. I'm not familiar with this project to tell how much work there would be to do it. And is there anything else simpler could be done to improve?

Update:
Another issue I found is that it fails silently without trying sbt credentials for private repositories, so we waited a long time and got nothing.

[julienrf]

Hello, thank you for explaining the issue. I agree that 140 seconds is way too long. A workaround could be to manually set versionPolicyPreviousVersions so that it is not computed at build-load-time.

It would also need to make settingKeys depending on it taskKeys.

Would that even be possible?

It take ~14 seconds for each subproject. […] Another issue I found is that it fails silently without trying sbt credentials for private repositories, so we waited a long time and got nothing.

It seems this is the heart of the issue. Unfortunately, because it is not possible to make a setting depend on task, we do not “inherit” from the standard sbt configuration to resolve dependencies, which would explain why your credentials are not taken into account. The only solution I see to this problem would be to introduce new settings to configure the dependency resolution engine here. From what I have seen in your draft PR, you only need to introduce a key to define the list of repositories to search in, which should have a default value corresponding to the current behavior. How does that sound? Please let me know if you would like more guidance.