Build repo specific PackageVersions.props w/dependencies declared in Version.Details.xml

[MichaelSimons]

Currently source-build build injects a PackageVersions.props into the repos builds that contains the current source-built package versions as well as all previously source built package versions. The result of this is that source-build does not behave the same as repo builds. Often times repos define Versions.props entries that are not auto-updated. They do not have a corresponding dependency in the Version.Details.xml file. Yet the way source-build works, these Versions.props entries are getting updated to use the source-built versions.

The solution to this is to build up a repo specific PackageVersions.props based on the dependencies defined in the Version.Details.xml. This will get source-build to build like the microsoft build and will be a forcing function to ensure the appropriate dependencies are defined (in order to eliminate pre-builts).

[MichaelSimons]

This is required for #2979

[oleksandr-didyk]

Update on progress -> was finishing implementation for a different task (#2655) + have a Bootcamp to attend this week, will properly start with the issue on Friday, 09/09/2022

[oleksandr-didyk]

Update on progress -> FR week + Hacathon next week, most probably will not produce meaningful progress in the next week

[oleksandr-didyk]

After initial investigation the current workload should probably be split into several separate issues due to the time and work required to accomplish a given milestone.

Based on introduction from @MichaelSimons and my own inspection, the steps taken should be as follows:

• add functionality for reading Version.Details.xml file and extracting dependency versions of declared projects -> Build-up list of used dependencies from Version.Details.xml #3040. This will allow us to collect only auto-updated dependencies while not interfering with dependency versions that the project has pinned.
• produce a project-specific PackageVersions.props file from the current and previous source-build versions -> Produce project-specific PackageVersions.props file #3041.
• document the changes introduced and PvP in general -> Document how modern ArPow PVP works, plus context #2546.
• opt-in all tarball-contributing repositories to the new version resolution system -> Enable individual PackageVersions.props file on tarball-contributing repositories #3043
• consume projects' PackageVersions.props files when source-building the entire product -> Consume project-specific PackageVersions.props file in end tarball build #3042.

Furthermore, there are several issues that are worth mentioning:

• repositories should utilize the latest versions of project dependencies (enforcement of this rule is not part of MVP)
  • having non-latest versions would require additional support from SB in terms of adding a ref pack for every version
  • maintaining and upgrading the pack would be costly
  • additional packs would bloat the size of the end tarball
  • SBRP in general should not contain any pre-release versions
• rollout of the proposed changes should be gradual
  • additional work would probably be required for on-boarding existing repositories to the new version handling (ex.: adding SBRPs or bumping dependencies to latest)
• tarball should be offline-buildable
  • PackageVersions.props files can be resolved with access to the feeds (meaning online) before the actual build is started

[MichaelSimons]

Once this functionality is picked up by each repo and within the tarball build, the existing infrastructure should be removed.

[MichaelSimons]

tarball should be offline-buildable
PackageVersions.props files can be resolved with access to the feeds (meaning online) before the actual build is started

This is certainly doable today with the current tarball creation process but becomes significantly more difficult with the VMR. We should discuss this at one of our syncs w/@premun and @crummel

[oleksandr-didyk]

Once this functionality is picked up by each repo and within the tarball build, the existing infrastructure should be removed.

Thank you for reminding me about it, I will create a separate issue for it since it can be achieved only after the repos are on the new system, which can take some time

[oleksandr-didyk]

tarball should be offline-buildable
PackageVersions.props files can be resolved with access to the feeds (meaning online) before the actual build is started

This is certainly doable today with the current tarball creation process but becomes significantly more difficult with the VMR. We should discuss this at one of our syncs w/@premun Premek Vysoky FTE and @crummel Chris Rummel FTE

Based on the info received on the sync, it seems that the PackageVersions.props files are resolved offline, so this might not be an issue anymore, correct?

[MichaelSimons]

@mmitche - was this completed with dotnet/installer#15267? I know we have the repo consumption side of this which is tracked in #3043.

[mmitche]

Yep, this is completed.