Specify all source-built dependencies and ensure they are regularly updated

[mmitche]

Overview

Source-build works today by building a repository, gathering the versions of the outputs, and creating a property file with those versions. This property file is then fed to downstream repositories, which import it after their eng/Versions.props. The resulting behavior is that the downstream repo overrides all dependency versions specified in eng/Versions.props with the versions built from source. This behavior is not the same as the Maestro dependency-flow based approach. Maestro only updates properties for dependencies that are specified in eng/Version.Details.xml.

This causes the following difference: Say that dotnet/aspnetcore has a Microsoft.Net.Compilers.Toolset dependency. It codes a property for that dependency's version in eng/Versions.props as MicrosoftNetCompilersToolsetVersion, with an older version of the package.

• When building via source-build, roslyn builds first, and aspnetcore will pick up the new version of Microsoft.Net.Compilers.Toolset, even though it doesn't want to.
• When building officially, the non-latest version will be used.

This difference tends to cause build breaks in source-build. The version bump may be significant and require repo reaction. This is not ideal and not sustainable.

One option would be to only override those versions that are specified in eng/Version.Details.xml. This would more closely align the source-build and current official builds. The huge downside is that this will cause an explosion of ref packs. And in some cases, we would be building against old versions but actually executing against newer ones. Non-ideal.

In summary, the goals are:

• Reduce source-build build breaks
• Avoid a large increase in ref packs.
• Ensure that it is possible to freeze on old versions of a specific component if absolutely necessary

Work Required

This work is dependent on dotnet/source-build#2982.

In this repo, do the following

• Obtain a report of un-specified dependencies in this repo. This issue will be updated with instructions on how to do so after .NET 8: Generate report of non-explicit dependencies for each repo in source build source-build#2982 is complete.
• Specify all dependencies classified as missing in the report in the eng/Version.Details.xml file.
• Upgrade dependencies regularly via Maestro dependency flow mechanisms, or pin dependencies that should stay the same, with justification.

[mmitche]

Closing as completed as part of the source-build sustainability (PVP and prebuilt detection) work