KoreBuild auto-update and version pinning

[mikeharder]

@anurse, @pakrym, and @mikeharder were discussing some changes to KoreBuild designed to improve the reliability of tests and developer builds:

• KoreBuild should automatically update itself if KoreBuild or the CLI has been updated. Currently, KoreBuild only updates after the ".build" folder is manually deleted.
• KoreBuild should pin the version of ASP.NET packages, so projects will reference 2.0.0-version rather than 2.0.0-*. This has several advantages:
  • Restore should never fail due to new packages being added to the feed
  • Run should never fail due to packages being newer than the installed shared runtime
  • Restore should be much faster
  • We should no longer need a proxy in front of MyGet
• The version of ASP.NET packages should not be updated until we've verified this version of all packages can be downloaded from MyGet

Developer command-line builds (build.ps1, build.sh) will always install the latest CLI (and shared runtime) ingested by ASP.NET, and pin the package version.

Developer VS builds will use the CLI and package versions installed by the last run of KoreBuild. While developing in VS, you may fall behind the latest version of CLI and packages, but things should never break.

Some of the CI builds (e.g. UniverseCoherence) may still need to use 2.0.0-* while building packages.

[analogrelay]

For the following:

KoreBuild should automatically update itself if KoreBuild or the CLI has been updated. Currently, KoreBuild only updates after the ".build" folder is manually deleted.

I couldn't find my code for conditional-GET in .sh and .ps1, but it wasn't too complicated. In PowerShell you've got Invoke-WebRequest which gives you a way to specify Request headers, and the response is an object containing the Response headers. With curl there's -H [HeaderName]: [HeaderValue] to set request headers and -D [FileName] to dump response headers to a file (which can be greped to get the ETag).

I assume, but haven't checked, that both have a timeout mechanism. So the startup flow is

1. Issue a GET with If-None-Match using previous Etag (stored in .build, if any) for latest KoreBuild zip with a very short timeout
2. If the Get returns a new file, unpack it and replace .build with the contents, also store the new Etag in .build
3. Continue as before

[Tratcher]

Is there an opt-out / override? The wildcards have been useful for other things like working on cross-repo changes. E.g. Build repo A, point repo B to repo A's artifacts and build B.

[kichalla]

@mikeharder and I were discussing about this scenario. Some options we can use(I like the global override too):

• https://github.com/aspnet/universe#verifying-cross-repo-changes (universe build locally should not use pinning that is being discussed here)
• Removing the $(AspNetCoreVersion) from B which is for the package reference of A and hardcoding to wildcard version (not pretty but should work)

[analogrelay]

We could also make it relatively simple to allow the AspNetCoreVersion property to be overridden, so it can be provided by either the command line, or a props file.

[analogrelay]

The props file would be more useful for Visual Studio work. We could have a crossrepo.props file conditionally imported (only if it exists) and gitignored, so that you can just drop it in place and set the version to whatever you want.

[kichalla]

Curious to know if there was any idea that has been discussed on how to implement this. Is there going to be a dependency.props file checked into Korebuild after every successful Coherence build and from there on any build would have this dependency.props file supplied to the repo with the expectation that the individual repos dependency props would get overwritten...also individual repos dependency.props file would need to get updated for VS scenarios.

[analogrelay]

Why would individual repo dependencies.props need to be updated? If KoreBuild supplied the props file and the projects imported it from there (possibly conditionally if we care about the scenario of opening in VS without KoreBuild), then it would be updated in VS as well. dependencies.props would be reserved for purely external dependencies (rather than dependencies that we produce as part of our build).

[kichalla]

(possibly conditionally if we care about the scenario of opening in VS without KoreBuild)

Yeah, I actually meant this scenario. If we do not want to initialize (to get the pinned dependency.props) via Korebuild and open VS, was wondering how do we get pinned versions to the repo (sure it need to be the dependency.props that a repo has and could be an imported file but this imported file which is in the repo still needs to be updated for the above VS scenario, right?)

[analogrelay]

I think we're already in a situation where KoreBuild is required to run properly in VS. If we did this, I think the way we'd do it is that dependencies.props keeps the 2.0.0-* version, but KoreBuild brings in a props file which overrides it. We can do this by importing the KoreBuild one first and making the AspNetCoreVersion property in the repo-local dependencies.props conditional on '$(AspNetCoreversion)' == ''.

This also leaves open the ability to set a fixed version in the repo-local dependencies.props by changing the value and making it unconditional.