Register dotnetup with BAR for channel-based publishing#54305
Register dotnetup with BAR for channel-based publishing#54305dsplaisted wants to merge 1 commit into
Conversation
Adds a publish_to_bar stage to the dotnetup CI pipeline that runs on every non-PR build. The stage: 1. Downloads the per-RID dotnetup-standalone-* pipeline artifacts. 2. Stages all 16 binary/.sha512 files flat under BlobArtifacts/ (Arcade's PublishArtifactsInManifestBase fetches blobs by basename only) and emits a v3 MergedManifest.xml whose Blob Id paths (dotnetup/<SemVer>/<file>) are used as the upload destinations. 3. Calls publish-build-assets.yml (Arcade's standard job) to register the build with the Build Asset Registry. Once dotnet/sdk@release/dnup is mapped as a default channel for 'dotnetup Daily' (maestro-configuration#61053) and the channel's TargetChannelConfig is registered (dotnet/arcade#16813), the Maestro Promotion Pipeline (dotnet#750) will publish each build's binaries to ci.dot.net/public/dotnetup/<SemVer>/ with stable aka.ms/dotnet/dotnetup/daily/<file> shortlinks. Other changes: - WriteDotnetupVersionFile (dotnetup.csproj): emits dotnetup.version.txt next to the published binary so the BAR staging job has an authoritative source for the Arcade-composed SemVer (no MSBuild file parsing in the pipeline). - dotnetup-executables.yml: copies dotnetup.version.txt into each per-RID standalone artifact. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Adds Build Asset Registry (BAR) registration for non-PR release/dnup dotnetup builds by staging standalone executables/checksums into the expected BlobArtifacts/ layout, generating a v3 MergedManifest.xml, and invoking Arcade’s standard publish-build-assets job so Maestro can later promote/upload to channel-based blob feeds.
Changes:
- Emit
dotnetup.version.txtduringdotnet publishso pipelines can consume the Arcade-computed$(Version)without parsing MSBuild files. - Include
dotnetup.version.txtin each per-RIDdotnetup-standalone-*pipeline artifact (Windows + Unix staging). - Add a new staging + BAR-registration job template and wire it into
.vsts-dnup-ci.ymlas apublish_to_barstage for non-PR builds.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
src/Installer/dotnetup/dotnetup.csproj |
Adds an MSBuild target to write dotnetup.version.txt into the publish output. |
eng/pipelines/templates/jobs/dotnetup/dotnetup-publish-to-bar.yml |
New job template that downloads per-RID standalone artifacts, stages a flat BlobArtifacts/ layout, generates MergedManifest.xml, and runs Arcade BAR registration. |
eng/pipelines/templates/jobs/dotnetup/dotnetup-executables.yml |
Updates per-RID staging to include dotnetup.version.txt in the standalone artifacts used by BAR staging. |
.vsts-dnup-ci.yml |
Adds the publish_to_bar stage (non-PR) that depends on all per-RID executable stages and runs the BAR registration template. |
| artifact: dotnetup-standalone-${{ entry.rid }} | ||
| path: $(Pipeline.Workspace)/standalone/${{ entry.rid }} |
| isWindows: true | ||
| - rid: win-arm64 | ||
| isWindows: true | ||
| - rid: linux-x64 | ||
| isWindows: false | ||
| - rid: linux-arm64 | ||
| isWindows: false | ||
| - rid: linux-musl-x64 | ||
| isWindows: false | ||
| - rid: linux-musl-arm64 | ||
| isWindows: false | ||
| - rid: osx-x64 | ||
| isWindows: false | ||
| - rid: osx-arm64 | ||
| isWindows: false |
|
Marking draft while we evaluate an alternative approach. Per Matt's feedback on the email thread, exploring switching dotnetup to use standard Arcade primitives (build.cmd -pack -sign -publish with /p:Projects=dotnetup.slnf) instead of the bespoke pipeline this PR builds on. Will pick this back up depending on how that exploration goes. |
Register dotnetup with BAR for channel-based publishing
Adds Build Asset Registry (BAR) registration to every non-PR dotnetup build, on top of the existing pipeline-artifact publishing. This is the prerequisite for shipping the dotnetup standalone executables to anonymous-access blob storage (
ci.dot.net/public/dotnetup/<SemVer>/) with stableaka.ms/dotnet/dotnetup/daily/<file>shortlinks.Tracks #54254.
What this PR does
Adds a new
publish_to_barstage to.vsts-dnup-ci.ymlthat runs on every non-PR build ofrelease/dnup. The stage is additive — it does not change anything about the existingdotnetup-standalone-*pipeline artifacts, which continue to be published exactly as today. It only:dotnetup-standalone-*pipeline artifact (8 RIDs) onto a Windows agent..sha512files flat underBlobArtifacts/. This flat layout is required because Arcade'sPublishArtifactsInManifestBase.DownloadFileAsyncfetches source blobs by basename only when the promotion pipeline runs.MergedManifest.xmlwhose<Blob Id="dotnetup/<SemVer>/<file>" />entries serve a dual purpose: the basename is the source path insideBlobArtifacts/, the full path is the destination layout under the channel's blob feed.publish-build-assets.ymljob, which registers the build with BAR.A small MSBuild target (
WriteDotnetupVersionFileindotnetup.csproj) emitsdotnetup.version.txtnext to each published binary so the staging job has an authoritative source for the Arcade-composed$(Version)SemVer (no MSBuild file parsing in the pipeline).End-to-end mechanics validated
Validated in two stages:
1. Experiment branch — promoted BAR build 314208 to General Testing (promotion build 2974290). All 16 binary/checksum assets landed at
https://ci.dot.net/dev/dotnetup/0.1.3-preview.4.26263.3/...with HTTP 200 anonymous access.2. This branch — BAR build 314269 (CI build 2974542) registered 17 assets at
dotnetup/0.1.3-preview.4.26263.5/...— identical structure to the experiment build, confirming the cleanups (squashed history, dropped manual gate, robust version-file lookup) didn't regress anything. Channel attachment deferred until dotnet/arcade#16813 merges.Asset list for build 314208:
dotnetup-win-x64.exedotnetup-win-arm64.exedotnetup-linux-x64dotnetup-linux-arm64dotnetup-linux-musl-x64dotnetup-linux-musl-arm64dotnetup-osx-x64dotnetup-osx-arm64(Plus matching
.sha512checksums.)Companion changes
This PR is the fourth in a coordinated series:
dotnetup Dailychanneldotnet/sdk@release/dnup → dotnetup DailyIf this PR merges before the arcade and default-channel-mapping PRs above, every CI build will register with BAR but no automatic publishing will happen — manual
darc add-build-to-channelis the workaround. Once all four are merged, every CI build ofrelease/dnupwill auto-publish toci.dot.net/public/dotnetup/<SemVer>/and refresh theaka.ms/dotnet/dotnetup/daily/<file>shortlinks.What's intentionally NOT in this PR
1ES.PublishNuget@1direct-push ofMicrosoft.Dotnet.Installationfor now. Including it here would mean either a publishing gap (if direct push is removed before automatic promotion is wired up) or double-publish (if both run simultaneously). Better as a follow-up PR sequenced after auto-promotion is confirmed working end-to-end.get-dotnetup.{sh,ps1}install scripts to anonymous URLs: the current scripts useazto download pipeline artifacts. Once builds are flowing toci.dot.net/public/dotnetup/...andaka.ms/dotnet/dotnetup/daily/...links are live, those scripts can drop theazrequirement. Follow-up PR.