Upgrade to V4 Arcade Publishing#439
Open
mmitche wants to merge 1 commit intodotnet:mainfrom
Open
Conversation
- Update Arcade SDK to 11.0.0-beta.26224.1 - Set PublishingVersion=4 in eng/Publishing.props - Add publishingVersion: 4 and enablePublishBuildAssets: true to jobs template - Add enablePublishing: true to Windows job - Remove enablePublishUsingPipelines and DotNetPublishUsingPipelines - Update publishingInfraVersion to 4 in post-build template Tracker: dotnet/arcade#16697 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Migrates this repo’s build/publishing infrastructure from Arcade V3 to V4 publishing, updates Arcade/common eng tooling, and moves projects forward to net9 to align with updated Arcade dependencies.
Changes:
- Switch publishing configuration/pipelines to Arcade Publishing V4 (including explicit MSI artifact publishing and
-publishbuild). - Update Arcade SDK/common eng templates & scripts (toolset acquisition, VMR sync improvements, pipeline artifact publishing changes).
- Update TFMs from
net8.0tonet9.0and bumpglobal.jsonSDK/runtime settings.
Reviewed changes
Copilot reviewed 75 out of 75 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| test/dotnet-core-uninstall.Tests/dotnet-core-uninstall.Tests.csproj | Move tests to net9.0. |
| test/dotnet-bootstrapper.Tests/dotnet-bootstrapper.Tests.csproj | Move tests to net9.0. |
| test/MSBuildBinLogQuery.Tests/MSBuildBinLogQuery.Tests.csproj | Move tests to net9.0. |
| src/redist/redist.csproj | Move project to net9.0. |
| src/dotnet-core-uninstall/dotnet-core-uninstall.csproj | Move project to net9.0. |
| src/dotnet-bootstrapper/dotnet-bootstrapper.csproj | Move project to net9.0. |
| src/MSBuildBinLogQuery/MSBuildBinLogQuery.csproj | Move project to net9.0. |
| global.json | Update SDK/Arcade versions and add 9.0.0 runtime. |
| eng/common/vmr-sync.sh | Add repo mapping lookup and fallback darc vmr reset behavior. |
| eng/common/vmr-sync.ps1 | Add repo mapping lookup, set DOTNET_ROOT, and fallback darc vmr reset. |
| eng/common/tools.sh | Toolset acquisition/layout updates; add dotnet wrapper; add warning controls and env isolation. |
| eng/common/tools.ps1 | Toolset acquisition/layout updates; add dotnet wrapper; remove xcopy MSBuild path; env isolation changes. |
| eng/common/templates/vmr-build-pr.yml | Pin VMR ref for PR build insertion. |
| eng/common/templates/variables/pool-providers.yml | Update example pool demand image. |
| eng/common/templates/steps/vmr-sync.yml | Remove manual VMR SHA checkout steps. |
| eng/common/templates/steps/component-governance.yml | Remove shim template. |
| eng/common/templates/jobs/codeql-build.yml | Remove shim template. |
| eng/common/templates/job/job.yml | Update job shim: CG handling changes; switch to pipeline artifacts publishing logic. |
| eng/common/templates-official/variables/sdl-variables.yml | Remove SDL variables file. |
| eng/common/templates-official/variables/pool-providers.yml | Update example pool image. |
| eng/common/templates-official/steps/publish-pipeline-artifacts.yml | Adjust sbom/isProduction parameter passthrough behavior. |
| eng/common/templates-official/steps/component-governance.yml | Remove official shim template. |
| eng/common/templates-official/jobs/codeql-build.yml | Remove official shim template. |
| eng/common/templates-official/job/job.yml | Update official job shim to publish pipeline artifacts and add V4 artifact publishing output. |
| eng/common/template-guidance.md | Update guidance to reflect removed shims/templates. |
| eng/common/sdl/trim-assets-version.ps1 | Remove SDL script. |
| eng/common/sdl/sdl.ps1 | Remove SDL script. |
| eng/common/sdl/run-sdl.ps1 | Remove SDL script. |
| eng/common/sdl/packages.config | Remove SDL packages config. |
| eng/common/sdl/init-sdl.ps1 | Remove SDL script. |
| eng/common/sdl/extract-artifact-packages.ps1 | Remove SDL script. |
| eng/common/sdl/extract-artifact-archives.ps1 | Remove SDL script. |
| eng/common/sdl/execute-all-sdl-tools.ps1 | Remove SDL script. |
| eng/common/sdl/configure-sdl-tool.ps1 | Remove SDL script. |
| eng/common/sdl/NuGet.config | Remove SDL NuGet config. |
| eng/common/sdk-task.sh | Update help text for new Arcade toolset layout. |
| eng/common/sdk-task.ps1 | Update SDK task execution to use VS MSBuild init (no xcopy MSBuild). |
| eng/common/renovate.env | Add shared env configuration for Renovate execution. |
| eng/common/post-build/sourcelink-validation.ps1 | Remove SourceLink validation script. |
| eng/common/post-build/redact-logs.ps1 | Use --source instead of --add-source for tool install. |
| eng/common/post-build/nuget-verification.ps1 | Add -UseBasicParsing to Invoke-WebRequest. |
| eng/common/native/install-dependencies.sh | Add ninja to dependencies; fix distro conditional formatting; adjust alpine packages. |
| eng/common/native/init-distro-rid.sh | Add OpenBSD RID support. |
| eng/common/internal-feed-operations.ps1 | Add -UseBasicParsing to Invoke-WebRequest. |
| eng/common/generate-sbom-prep.sh | Remove SBOM prep script. |
| eng/common/generate-sbom-prep.ps1 | Remove SBOM prep script. |
| eng/common/darc-init.sh | Use --source instead of --add-source for tool install. |
| eng/common/darc-init.ps1 | Use --source instead of --add-source for tool install. |
| eng/common/cross/toolchain.cmake | Add OpenBSD cross-compile detection/triples. |
| eng/common/cross/build-rootfs.sh | Update FreeBSD versions; add OpenBSD sysroot creation logic/packages. |
| eng/common/core-templates/steps/source-index-stage1-publish.yml | Fix parameter casing for package source. |
| eng/common/core-templates/steps/source-build.yml | Mark logs as non-production via isProduction. |
| eng/common/core-templates/steps/publish-logs.yml | Switch to pipeline artifacts; adjust artifact naming; remove MaestroAccessToken from redaction inputs. |
| eng/common/core-templates/steps/install-microbuild.yml | Remove @self template resource qualifier. |
| eng/common/core-templates/steps/install-microbuild-impl.yml | Fix boolean comparison for enablePreviewMicrobuild. |
| eng/common/core-templates/steps/generate-sbom.yml | Deprecate SBOM generation step template. |
| eng/common/core-templates/steps/component-governance.yml | Remove CG step template. |
| eng/common/core-templates/stages/renovate.yml | Add reusable Renovate pipeline stage template. |
| eng/common/core-templates/post-build/setup-maestro-vars.yml | Switch ReleaseConfigs download to pipeline artifacts. |
| eng/common/core-templates/post-build/post-build.yml | Add V4 support for validation downloads; remove SourceLink validation; update pools/images; adjust publishing flow. |
| eng/common/core-templates/post-build/common-variables.yml | Remove SourceLink CLI version variable. |
| eng/common/core-templates/jobs/jobs.yml | Add publishingVersion parameter plumbing. |
| eng/common/core-templates/jobs/codeql-build.yml | Remove CodeQL jobs template. |
| eng/common/core-templates/job/source-index-stage1.yml | Skip component governance; update pool images. |
| eng/common/core-templates/job/source-build.yml | Update pool images/demands. |
| eng/common/core-templates/job/renovate.yml | Add reusable Renovate job template. |
| eng/common/core-templates/job/publish-build-assets.yml | Update pools/images; publish manifests/configs via pipeline artifacts; pass StageLabel to publish-logs. |
| eng/common/core-templates/job/onelocbuild.yml | Update pools/images; add xLocCustomPowerShellScript support. |
| eng/common/core-templates/job/job.yml | Add enablePublishing; remove componentGovernanceSteps injection. |
| eng/common/build.sh | Add --warnNotAsError option plumbed through to MSBuild invocation. |
| eng/common/build.ps1 | Add -warnNotAsError parameter and usage docs. |
| eng/Version.Details.xml | Update Arcade SDK dependency version/SHA. |
| eng/Publishing.props | Switch to Publishing V4; disable default artifacts; declare MSI artifact explicitly. |
| .vsts-pr.yml | Update PR pipeline to Publishing V4 infra parameter usage and remove obsolete flags. |
| .vsts-ci.yml | Update official pipeline for V4 publishing and enable publishing on Windows job. |
Comments suppressed due to low confidence (1)
eng/common/templates/job/job.yml:18
- This shim declares legacy parameters (disableComponentGovernance/enableSbom/PackageVersion/BuildDropPath) but then forwards all parameters (except steps/is1ESPipeline) into /eng/common/core-templates/job/job.yml. The core template doesn't define these parameters, so pipeline compilation will fail with an "Unexpected parameter" error. Exclude these legacy keys from the forwarding loop (or remove them entirely) before passing parameters through.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| arguments: > | ||
| -BuildId $(BARBuildId) | ||
| -PublishingInfraVersion ${{ parameters.publishingInfraVersion }} | ||
| -PublishingInfraVersion 3 |
There was a problem hiding this comment.
publish-using-darc.ps1 is always invoked with -PublishingInfraVersion 3, ignoring the publishingInfraVersion parameter (which now allows 4). If V4 is selected, this will still force V3 publishing behavior. Pass through the parameter value (or remove the parameter if Darc publishing is intentionally fixed to V3).
Suggested change
| -PublishingInfraVersion 3 | |
| -PublishingInfraVersion ${{ parameters.publishingInfraVersion }} |
Comment on lines
+145
to
+157
| - ${{ if eq(parameters.publishingInfraVersion, 4) }}: | ||
| - task: DownloadPipelineArtifact@2 | ||
| displayName: Download Pipeline Artifacts (V4) | ||
| inputs: | ||
| itemPattern: '*/packages/**/*.nupkg' | ||
| targetPath: '$(Build.ArtifactStagingDirectory)/PipelineArtifactsDownload' | ||
| - task: CopyFiles@2 | ||
| displayName: Flatten packages to PackageArtifacts | ||
| inputs: | ||
| SourceFolder: '$(Build.ArtifactStagingDirectory)/PipelineArtifactsDownload' | ||
| Contents: '**/*.nupkg' | ||
| TargetFolder: '$(Build.ArtifactStagingDirectory)/PackageArtifacts' | ||
| flattenFolders: true |
There was a problem hiding this comment.
In the V4 path, DownloadPipelineArtifact@2 is missing the inputs that target the specific build resolved by setup-maestro-vars (AzDOProjectName/AzDOPipelineId/AzDOBuildId). As written, it will download artifacts from the current pipeline run, which breaks post-build validation/promotion scenarios when BARBuildId points at a different build. Configure DownloadPipelineArtifact to download from the specific pipeline run ID (or otherwise use the resolved variables).
Comment on lines
+203
to
+214
| - task: DownloadPipelineArtifact@2 | ||
| displayName: Download Pipeline Artifacts (V4) | ||
| inputs: | ||
| itemPattern: '*/packages/**/*.nupkg' | ||
| targetPath: '$(Build.ArtifactStagingDirectory)/PipelineArtifactsDownload' | ||
| - task: CopyFiles@2 | ||
| displayName: Flatten packages to PackageArtifacts | ||
| inputs: | ||
| SourceFolder: '$(Build.ArtifactStagingDirectory)/PipelineArtifactsDownload' | ||
| Contents: '**/*.nupkg' | ||
| TargetFolder: '$(Build.ArtifactStagingDirectory)/PackageArtifacts' | ||
| flattenFolders: true |
There was a problem hiding this comment.
Same issue as the NuGet validation job: the V4 DownloadPipelineArtifact@2 step doesn't specify project/pipeline/run information, so it cannot download artifacts for the AzDOBuildId selected by setup-maestro-vars when promoting/validating an external BAR build. Update the task inputs to target the resolved build (AzDOProjectName/AzDOPipelineId/AzDOBuildId) for V4 as well.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Upgrade to V4 Arcade Publishing
This PR migrates dotnet/cli-lab from Arcade V3 publishing to V4 publishing.
Changes
MSBuild (�ng/Publishing.props):
PublishingVersionfrom 3 to 4EnableDefaultArtifactsto false (repo produces MSI, not NuGet packages)Artifactitem withRelativeBlobPathOfficial pipeline (
.vsts-ci.yml):enablePublishUsingPipelinesenablePublishBuildAssets: trueandpublishingVersion: 4enablePublishing: trueon Windows job-publishto build command for asset manifest generationDotNetPublishUsingPipelinesfrom_OfficialBuildArgsPR pipeline (
.vsts-pr.yml):enablePublishUsingPipelinesDotNetPublishUsingPipelinesfrom_OfficialBuildArgspublishingInfraVersionfrom 3 to 4Arcade SDK update (via
darc update-dependencies):TFM updates (required by new xunit v3 from Arcade):
Validation tracking
--skip-assets-publishing)dotnet-core-uninstall.msi@ 1.7.722502) plus merged manifest and PDB symbols. Merged manifest confirmedPublishingVersion="4".