[build] Enable signing for .NET 6 artifacts by pjcollins · Pull Request #5529 · dotnet/android

pjcollins · 2021-01-21T01:05:06Z

Context: https://github.com/xamarin/yaml-templates/tree/564100f2aa923ee4db4b69cc45a622822643309b/sign-artifacts
Context: https://devdiv.visualstudio.com/DevDiv/_wiki/wikis/DevDiv.wiki/650/Enabling-Signing-in-Your-Projects

The structure of our YAML pipeline has been updated to enable signing
for all relevant .NET 6 artifacts. The new pipeline hierarchy is
described below:

mac_build
    ...
    sign nupkgs

        create net6 .pkg
            download signed .nupkgs
            push signed .nupkgs to feed
            harden/sign mach-o binaries
            create .pkg
            sign .pkg
            notarize .pkg
            publish .pkg

            create net6 .msi
                download signed .nupkgs
                download notarized .pkg
                create and sign .msi
                upload .msi and .pkg to blob storage

The dotnet_installers stage now starts with a templated job which will
sign all .nupkg files created by our mac_build stage. These signed
packages are then downloaded by a new macOS job that will push them to a
feed and use them to create, sign, and notarize the .NET 6 .pkg file.
Finally, the job which previously created our .NET 6 .msi has been
updated to wait for the result of the .NET 6 .pkg creation job. This
job will now consume the signed .nupkg files to produce a signed .msi,
and upload both the signed .pkg and .msi files to blob storage.

None of the MicroBuild tasks/targets will run during local builds, so
we do not need to worry about additional environment provisioning when
creating these installers locally. In order to test locally, the MicroBuild
signing NuGet needs to be installed locally and either the
MicroBuildOverridePluginDirectory or MicroBuildPluginDirectory
properties must be set and point to the folder where the signing NuGet
was installed. See the "Enabling signing in your projects" link above
for more information.

The new SignList.xml file is used by the .nupkg signing job. The content
of this file controls which certs are used to sign which files. Anything that
was signed through an alternate source should be added to the @(Skip)
ItemGroup. Any third party file that needs signing should be added to the
@(ThirdParty) ItemGroup.

Context: https://github.com/xamarin/yaml-templates/tree/564100f2aa923ee4db4b69cc45a622822643309b/sign-artifacts Context: https://devdiv.visualstudio.com/DevDiv/_wiki/wikis/DevDiv.wiki/650/Enabling-Signing-in-Your-Projects The structure of our YAML pipeline has been updated to enable signing for all relevant .NET 6 artifacts. The new pipeline hierarchy is described below: mac_build ... sign nupkgs create net6 .pkg download signed .nupkgs push signed .nupkgs to feed harden/sign mach-o binaries create .pkg notarize .pkg publish .pkg create net6 .msi download signed .nupkgs download notarized .pkg create and sign .msi upload .msi and .pkg to blob storage The `dotnet_installers` stage now starts with a templated job which will sign all .nupkg files created by our `mac_build` stage. These signed packages are then downloaded by a new macOS job that will push them to a feed and use them to create, sign, and notarize the .NET 6 .pkg file. Finally, the job which previously created our .NET 6 .msi has been updated to wait for the result of the .NET 6 .pkg creation job. This job will now consume the signed .nupkg files to produce a signed .msi, and upload both the signed .pkg and .msi files to blob storage.

pjcollins · 2021-01-21T01:06:28Z

Commit 5f79468 has successfully signed artifacts - https://dl.internalx.com/vsts-devdiv/Xamarin.Android/public/net6/4389120/sign-net6-nupkg/5f794681a045dad295119f3ea422beb1032b727a/artifacts.json. This status might get overwritten when a PR build runs however. This should almost be ready to go once https://github.com/xamarin/yaml-templates/pull/96 is merged, assuming test signing works as expected against a PR build.

jonathanpeppers

Otherwise looks great! I tested the Windows .msi and it is signed!

jonathanpeppers · 2021-01-21T03:40:49Z

    <_MSBuildFiles Include="$(MSBuildSrcDir)\Xamarin.Installer.Common.dll" />
    <_MSBuildFiles Include="$(MSBuildSrcDir)\Xamarin.Installer.Common.pdb" />
    <_MSBuildFiles Include="$(MSBuildSrcDir)\Xamarin.Installer.Common.targets" />
-    <_MSBuildFiles Include="$(MSBuildSrcDir)\K4os.Compression.LZ4.dll" />


Were the two assemblies here listed twice?

Yeah, I had an initial take where I was marking third party items in this file and found a couple duplicates that should be cleaned up as part of that.

pjcollins · 2021-01-22T17:28:07Z

/azp run

azure-pipelines · 2021-01-22T17:28:50Z

Azure Pipelines successfully started running 1 pipeline(s).

pjcollins · 2021-01-22T21:04:43Z

@trevors20 The az-client-update template hit an issue on this latest run - https://devdiv.visualstudio.com/DevDiv/_build/results?buildId=4397974&view=logs&jobId=a62e275d-46d5-5c3b-5a47-c4e00ae7427d&j=a62e275d-46d5-5c3b-5a47-c4e00ae7427d&t=58b91e55-0185-54d7-8ced-025ce63f4b7f.

Error: No similarly named formulae found.
Error: No available formula or cask with the name "builder/azure-cli/azure-cli@2.8.0".
==> Searching for a previously deleted formula (in the last month)...
Error: No previously deleted formula found.
brew link azure-cli@2.8.0
Error: No such keg: /usr/local/Cellar/azure-cli@2.8.0

It looks like this is also being worked on in #5509, maybe I should wait for that to land first?

pjcollins · 2021-01-25T19:06:26Z

Unsigned PR artifacts from commit c90b97b:
https://dl.internalx.com/vsts-devdiv/Xamarin.Android-PR/public/net6/4404211/merge/329563c7657077b99ed10bfd7a5848ea8751b8b7/artifacts.json

Final test build for real signing: https://devdiv.visualstudio.com/DevDiv/_build/results?buildId=4404565&view=results
Signed PR artifacts from commit c90b97b:
https://dl.internalx.com/vsts-devdiv/Xamarin.Android/public/net6/4404565/sign-net6-nupkg/c90b97b35a15a210c090605010f2a06522ef2347/artifacts.json

This should be ready to merge.

jonathanpeppers

Otherwise LGTM 👍

Context: #5529 Commit dc1f7ba enabled ESRP signing for all .NET 6 artifacts, and we've been asked to apply these changes to our stable .pkg installer that is still using a deprecated Apple Developer account and signing logic. The mac build job has been updated to sign and harden all Mach-O files included in both our stable and .NET 6 .pkg files. This was previously happening in two different passes, the first for the stable .pkg content during the build job and the second for the .NET 6 .pkg content during the `dotnet_create_pkg` job. Signing of the stable .pkg has been migrated to use the ESRP signing service via the tasks/targets imported by the new MicroBuild package reference. Notarization of the stable .pkg has been moved to the mac build job. As a result, all legacy macOS automated tests will now run against the final version of the package we ship. This will allow us to catch any potential issues with notarization earlier.

Context: #5529 Commit dc1f7ba enabled ESRP signing for all .NET 6 artifacts, and we've been asked to apply these changes to our stable `.pkg` installer which is using a deprecated Apple Developer account and signing logic. The macOS build job has been updated to sign and harden all Mach-O files included in both our stable and .NET 6 `.pkg` files. This was previously happening in two different passes, the first for the stable `.pkg` content during the build job and the second for the .NET 6 `.pkg` content during the `dotnet_create_pkg` job. Signing of the stable `.pkg` has been migrated to use the ESRP signing service via the tasks/targets imported by the new MicroBuild package reference. Our previous notarization preparation logic required us to unpack `bundletool.jar`, and sign the version of `aapt2` that is bundled inside. We now instead remove all embedded versions of `aapt2`, as we pass in the path our own bundled `aapt2` via `bundletool.jar --aapt2 /path/to/our/aapt2`.

pjcollins and others added 8 commits January 13, 2021 22:58

Sign and notarize pkg

68f5649

Fix some paths

1df96d2

Fix another path

e2b5aff

Use signing dependencies template

8d113c7

Upload artifacts, set override path prop

c4e43db

Try to zip .pkg for signing

4c934db

Fix net6-pkg artifact name, remove signing plugin

5f79468

pjcollins requested review from jonathanpeppers, mjbond-msft and trevors20 January 21, 2021 01:07

jonathanpeppers reviewed Jan 21, 2021

View reviewed changes

Apply feedback

31ae7c9

trevors20 reviewed Jan 21, 2021

View reviewed changes

Comment thread build-tools/automation/yaml-templates/install-signing-plugin.yaml Outdated

trevors20 approved these changes Jan 21, 2021

View reviewed changes

pjcollins added 3 commits January 21, 2021 13:30

Use template for az version checking

0e4c3d1

Merge remote-tracking branch 'origin/master' into sign-net6-nupkg

1f5b90b

Fix typo

3c5555a

pjcollins requested a review from jonpryor January 21, 2021 18:53

pjcollins added 2 commits January 22, 2021 12:32

Track main branch of templates again

930a831

Track main branch of templates for signing as well

9d01bea

pjcollins added 3 commits January 25, 2021 10:07

Merge remote-tracking branch 'origin/master' into sign-net6-nupkg

f33a884

Do not attempt to test sign .pkg

ecd927a

Don't install tooling if previous step failed

c90b97b

pjcollins marked this pull request as ready for review January 25, 2021 19:08

jonathanpeppers approved these changes Jan 25, 2021

View reviewed changes

Comment thread build-tools/automation/azure-pipelines.yaml Outdated

apply feedback

58aecd8

jonpryor merged commit dc1f7ba into master Jan 26, 2021

jonpryor deleted the sign-net6-nupkg branch January 26, 2021 21:22

pjcollins mentioned this pull request Mar 5, 2021

[build] Migrate stable .pkg signing to ESRP #5700

Merged

github-actions Bot locked and limited conversation to collaborators Jan 24, 2024

Conversation

pjcollins commented Jan 21, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pjcollins commented Jan 21, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jonathanpeppers left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

jonathanpeppers Jan 21, 2021

Choose a reason for hiding this comment

Uh oh!

pjcollins Jan 21, 2021

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

pjcollins commented Jan 22, 2021

Uh oh!

azure-pipelines Bot commented Jan 22, 2021

Uh oh!

pjcollins commented Jan 22, 2021

Uh oh!

pjcollins commented Jan 25, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jonathanpeppers left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

pjcollins commented Jan 21, 2021 •

edited

Loading

pjcollins commented Jan 21, 2021 •

edited

Loading

pjcollins commented Jan 25, 2021 •

edited

Loading