Automate Versions of Packages

[michael-hawker]

Problem Statement

Each experiment's version number is set by the template and never gets incremented. Therefore every time we merge a PR to main, the packages for all experiments are being rebuilt and repushed to overwrite prior packages in our DevOps feed.

Expected Behavior

We want to incrementally increase the version number of new packages as we create new builds (and only keep a rolling set of packages, this should be pre-configured in DevOps already).

Ideally, we should only update and push packages for merges to main that contain code that has changed for those experiments, but determining that extra level of optimization would require more work.

At minimum though, we should append the build number to packages so we get a unique NuGet package for experiments. Individual experiment owners can increment their library version number as they deem appropriate otherwise beyond that though to help indicate breaking changes.

[Arlodotexe]

I know Azure DevOps provides a unique build number each run, but if GitHub doesn't then we can always create one based on the current date/time.

Ideally, we should only update and push packages for merges to main that contain code that has changed for those experiments, but determining that extra level of optimization would require more work.

It just happens that I did this for a different project within the last week.

If we're able to tag the latest commit that each package is created against, then we can use the following PowerShell script

$tagName = "theTagName"
$pathToSample = "/path/to/relevant/sample/"

# Output all the commits between previous tag and current head
#  --pretty=format:'%h' outputs only the hash, forcing one line per commit
#  -- ":/path/to/relevant/sample/" tells git to only look in the provided folder
$res = Invoke-Expression "git log $tagName...HEAD --pretty=format:'%h'  -- `":$pathToSample`""

$hasNoChanges = $null -eq $res -or $res.length -eq 0;

See pathspec in the gitglossary for more info.

[michael-hawker]

Right now, the package will skip updating silently, unless the version is manually revved. (see build here)

Run dotnet nuget push "**/*.nupkg" --api-key dummy --source LabsFeed --skip-duplicate
Pushing CommunityToolkit.Labs.Uwp.CanvasLayout.0.0.1.nupkg to 'https://pkgs.dev.azure.com/dotnet/[6](https://github.com/CommunityToolkit/Labs-Windows/runs/6613046488?check_suite_focus=true#step:12:7)96bc9fd-f160-4e97-a1bd-7cbbb3b58f66/_packaging/9c12ba9d-e4eb-4290-9266-e0fb1a871009/nuget/v2/'...
  PUT https://pkgs.dev.azure.com/dotnet/696bc9fd-f160-4e97-a1bd-7cbbb3b58f66/_packaging/9c12ba9d-e4eb-4290-9266-e0fb1a871009/nuget/v2/
  Conflict https://pkgs.dev.azure.com/dotnet/696bc9fd-f160-4e9[7](https://github.com/CommunityToolkit/Labs-Windows/runs/6613046488?check_suite_focus=true#step:12:8)-a1bd-7cbbb3b58f66/_packaging/9c12ba9d-e4eb-4290-9266-e0fb1a871009/nuget/v2/ 3315ms
Package 'D:\a\Labs-Windows\Labs-Windows\labs\CanvasLayout\src\bin\Package\CommunityToolkit.Labs.Uwp.CanvasLayout.0.0.1.nupkg' already exists at feed 'https://pkgs.dev.azure.com/dotnet/696bc9fd-f160-4e97-a1bd-7cbbb3b58f66/_packaging/9c12ba9d-e4eb-4290-9266-e0fb1a871009/nuget/v2/'.
Pushing CommunityToolkit.Labs.Uwp.SizerBase.0.0.1.nupkg to 'https://pkgs.dev.azure.com/dotnet/696bc9fd-f160-4e97-a1bd-7cbbb3b58f66/_packaging/9c12ba9d-e4eb-4290-9266-e0fb1a871009/nuget/v2/'...
  PUT https://pkgs.dev.azure.com/dotnet/696bc9fd-f160-4e97-a1bd-7cbbb3b58f66/_packaging/9c12ba9d-e4eb-4290-9266-e0fb1a871009/nuget/v2/
  Conflict https://pkgs.dev.azure.com/dotnet/696bc9fd-f160-4e97-a1bd-7cbbb3b5[8](https://github.com/CommunityToolkit/Labs-Windows/runs/6613046488?check_suite_focus=true#step:12:9)f66/_packaging/9c12ba9d-e4eb-4290-9266-e0fb1a871009/nuget/v2/ 332ms
Package 'D:\a\Labs-Windows\Labs-Windows\labs\SizerBase\src\bin\Package\CommunityToolkit.Labs.Uwp.SizerBase.0.0.1.nupkg' already exists at feed 'https://pkgs.dev.azure.com/dotnet/696bc9fd-f160-4e97-a1bd-7cbbb3b58f66/_packaging/9c12ba9d-e4eb-4290-9266-e0fb1a871009/nuget/v2/'.

[michael-hawker]

Pointer to current package (note duplicated between WinUI 2 / 3 jobs):

Labs-Windows/.github/workflows/build.yml

Lines 58 to 60 in f8620b9

	- name: pack experiments
	working-directory: ./common/Scripts/
	run: ./PackEachExperiment.ps1 all

https://github.com/CommunityToolkit/Labs-Windows/blob/main/common/Scripts/PackEachExperiment.ps1

[michael-hawker]

Matt pointed to the Template Studio path syntax here:
https://github.com/microsoft/TemplateStudio/blob/main/_build/pipelines/ci-Shared.yml#L9

Paths are part of triggers, don't know about at the job level would have to dig into the docs: https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#onpushpull_requestpull_request_targetpathspaths-ignore

Also found this:
https://github.community/t/run-job-only-if-folder-changed/118292/2

[michael-hawker]

Uno is using GitVersion, so something to look at.

[michael-hawker]

@Arlodotexe should we move this issue to the tooling submodule repo?

Also found this GitHub action which looks promising for package pushing only changed packages: https://github.com/marketplace/actions/changed-files

[michael-hawker]

Will leave this open to track migrating Labs over, but the new setup from the main repo is looking pretty slick. Handles most of our needs.

Main limitation is we can't move more than one-official nuget build a day as we're using the date as the revision number with our new schema. But we have our main builds all numbered so that's fine, same with PR builds, so it's pretty slick.

Other main hurdle which is orthogonal to this is how we release only the updated/sub-set of packages. That matters more for Labs-Windows as they're all independent, we've noticed with the main repo the inter-dependencies on Extensions (mostly among other packages) means that it's usually better to ship them all in lock-step. Unwinding those dependencies changes to only ship things which depend on what's being changed or depend on it would be pretty hard...