azure-pipelines: port build-git-installers to Azure Pipelines (Linux + macOS + Windows)#910
Draft
azure-pipelines: port build-git-installers to Azure Pipelines (Linux + macOS + Windows)#910
Conversation
Port the build-dependency setup from the GitHub workflow's
create-linux-unsigned-artifacts job
(.github/workflows/build-git-installers.yml). The package list
matches one for one: build-essential for the C toolchain,
tcl tk for git-gui, gettext for i18n, asciidoc and xmlto for
documentation, libcurl4-gnutls-dev / libpcre2-dev / zlib1g-dev /
libexpat-dev for the Git build flags this pipeline will use, and
curl / ca-certificates for any in-job downloads.
The GitHub workflow runs all of this inside an ubuntu:20.04 /
ubuntu:22.04 container, both to pin the resulting .deb's glibc
ABI floor and to give apt-get a root-owned filesystem. The 1ES
pool images we run on
(GitClientPME-1ESHostedPool-{intel,arm64}-pc) silently ignore a
job-level `container:` directive, so the build executes on the
bare Ubuntu host VM as the unprivileged agent user. Run apt-get
via `sudo`, and to at least pin the .deb's glibc floor to
something an audit can read back, log the running Ubuntu version,
kernel, and effective UID at the start of the job.
Two pieces of the workflow's setup are intentionally left out:
The DEBIAN_FRONTEND=noninteractive / TZ=Etc/UTC env vars exist
only to keep `tzdata` from prompting interactively when it gets
pulled in inside the container (see 842cfa4 (fixup!
release: build unsigned Ubuntu .deb package, 2025-02-13)); the
bare 1ES image already has tzdata configured and they would have
no effect.
The Node.js workaround exists to satisfy GitHub Actions' Node-based
shim, which Azure Pipelines does not need.
Re-introducing a real container later (whether via 1ES's container
option, a custom container job, or a build inside docker invoked
from a step) is a separate question.
Assisted-by: Claude Opus 4.7
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Replace the Linux dummy build with the real package build, ported from the create-linux-unsigned-artifacts job in .github/workflows/build-git-installers.yml. The pipeline already wires $(git_version) from the prereqs stage, so the version no longer comes from a tag_version job output. $(deb_arch) is taken straight from the matrix entry, dropping the GitHub workflow's runtime dpkg-architecture round-trip; the matrix already names amd64 and arm64 explicitly. Parallelism switches from the workflow's hard-coded -j5 (a runner-specific holdover) to -j$(nproc), which is the common default and adapts to whatever the 1ES pool gives us. The shell prologue switches from `set -ex` to `set -euo pipefail` so an unbound variable or a failed step in a pipeline aborts the job rather than silently producing a broken .deb. The make recipe and DEBIAN/control body match the workflow byte for byte: same DESTDIR layout, same Make flags (USE_LIBPCRE, USE_CURL_FOR_IMAP_SEND, NO_OPENSSL, NO_CROSS_DIRECTORY_HARDLINKS, ASCIIDOC8, ASCIIDOC_NO_ROFF, ASCIIDOC='TZ=UTC asciidoc'), same prefix and gitexecdir/libexecdir/htmldir, same install targets, and the same Depends list and Description text. The only intentional content change is the package version and architecture fields, which now come from $(git_version) and $(deb_arch). The sed 's/-rc/.rc/g' substitution carried over because Git's GIT-VERSION-GEN expects rc tags spelled with a dot. Output goes under $(Build.ArtifactStagingDirectory)/app/, where the existing ESRP signing template will pick it up via its '**/*.deb' pattern. The collect step below still uses the dummy 'cp -R app/* _final/' shape; tightening that is the next commit. Assisted-by: Claude Opus 4.7 Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Replace the Linux dummy collect step with a focused move of just the signed microsoft-git_<version>_<arch>.deb into $(Build.ArtifactStagingDirectory)/_final/, which the existing templateContext.outputs.pipelineArtifact already publishes as the linux_x64 / linux_arm64 artifact. The dummy version copied everything under app/* into _final/, which worked in isolation but would have started silently uploading any intermediate files (including the pkgroot/ staging tree the build step now writes alongside the .deb). Naming the file precisely also turns "ESRP signed something else" into a missing-file error rather than a silent wrong-artifact upload. Assisted-by: Claude Opus 4.7 Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Port the dual-architecture Homebrew install from the GitHub workflow's create-macos-artifacts job (.github/workflows/build-git-installers.yml). The native Homebrew on the macOS-latest pool image is arm64 and lives at /opt/homebrew. To produce a universal binary we additionally need the x86_64 build of gettext/libintl, so install a separate x86_64 Homebrew under /usr/local via the upstream installer running under Rosetta and pull gettext from there as well. The native arm64 brew installs the rest of the build chain: automake, asciidoc, xmlto, and docbook (the same set the workflow installs). The two arch-specific libintl.a copies are then combined with lipo into a universal archive at the workspace root, where the upcoming config.mak's LDFLAGS = -L"$(pwd)" will find it. libintl depends on iconv, but the system /usr/lib/libiconv.dylib is already universal and exports the _iconv* symbols Homebrew's gettext was built against; Homebrew's own libiconv exports _libiconv* and would not link, which is why the comment from the workflow is preserved here. Assisted-by: Claude Opus 4.7 Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Port the config.mak generation from the GitHub workflow's create-macos-artifacts job. With the universal libintl.a from the previous step in place, the build needs the Make flags that turn on the dual-arch compile and that route around several macOS quirks: HOST_CPU=universal, dual -arch CFLAGS (the actual universal-binary driver), -DNO_OPENSSL for contrib Makefiles that do not see the main Makefile's NO_OPENSSL handling, and explicit USE_HOMEBREW_LIBICONV/ICONVDIR overrides so we link against the universal /usr/lib/libiconv.dylib (whose unprefixed _iconv* symbols are what Homebrew's gettext was built against, unlike Homebrew's own libiconv with its prefixed _libiconv* symbols). CFLAGS adds the gettext include dirs from both Homebrew prefixes; LDFLAGS = -L"$(pwd)" so the linker finds the universal libintl.a in the workspace root. CURL_LDFLAGS / CURL_CONFIG pin against the OS-supplied libcurl rather than a Homebrew copy. Finally, SKIP_DASHED_BUILT_INS disables the dashed built-ins; on macOS the hard-link optimisation does not kick in for the staging tree and the resulting full copies would bloat the eventual .dmg. Compared with the GitHub workflow, the differences are mechanical: the source tree is at $(Build.SourcesDirectory) (no actions/checkout "path: git" subdir), so config.mak and the version file land at the worktree root; $(git_version) replaces tag_version; and the script prologue is set -euo pipefail rather than set -ex. Assisted-by: Claude Opus 4.7 Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Port the build sequence from the GitHub workflow's create-macos-artifacts step. Runs `make GIT-VERSION-FILE dist dist-doc` in the source tree, recovers the original commit OID from the resulting source tarball with `git get-tar-commit-id` (this becomes GIT_BUILT_FROM_COMMIT, which the macos-installer Makefile bakes into `git version --build-options`), then extracts the source and manpage tarballs into payload/ and manpages/, drops a copy of the worktree config.mak inside the extracted source so the universal-build flags apply during the real compile, and finally runs `make -C .github/macos-installer payload` to produce the universal binary tree. The macos-installer Makefile derives BUILD_DIR from $(GITHUB_WORKSPACE), which is unset under ADO. Setting GITHUB_WORKSPACE=$(Build.SourcesDirectory) in the task env block gives the Makefile the same anchor it had under GitHub Actions, so payload/git-$VERSION/ ends up where the targets expect it. XML_CATALOG_FILES is exported to the catalogs from the Homebrew docbook installed in the dependencies step, so asciidoc/xmlto can resolve their DTDs during dist-doc. VERSION is exported because the macos-installer Makefile reads it directly. The .rc/-rc rewrite stays consistent with the config-generation step: the on-disk version file has the .rc form (Git's GIT-VERSION-GEN convention), while the package and artifact filenames keep the original tag spelling (which is what the Makefile's ORIGINAL_VERSION tracks). Assisted-by: Claude Opus 4.7 Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Replace the dummy "Hello, Mac!" build placeholder and the dummy ESRP signing block with a real signing flow against the universal payload built in the previous step. The pattern follows git-credential-manager/.azure-pipelines/release.yml: pre-filter the payload to just the Mach-O files (using `file --mime` matching `mach`, the same heuristic the existing .github/scripts/codesign.sh uses), copy that subset into a staging directory under $(Build.ArtifactStagingDirectory)/macos-tosign/ preserving relative paths, hand the staging dir off to the existing .azure-pipelines/esrp/sign.yml template (which zips, signs via EsrpCodeSigning@6 with KeyCode CP-401337-Apple + OperationCode MacAppDeveloperSign + Hardening enabled, and extracts back into the staging dir), then copy the signed binaries back over the payload tree. The pre-filter is necessary because the existing template's CopyFiles@2 step uses minimatch globs and the only reliable way to pick out Mach-O files is by file content. Signing the entire payload tree would either fail on non-binary files or sign things that should not be signed (shell scripts, perl, manpages, templates, the uninstall.sh). UseDotNet@2 (8.x) installs the .NET SDK that EsrpCodeSigning@6 depends on; the macOS-latest pool image does not provide it. The whole block is gated on the `esrp` pipeline parameter, matching the existing convention in the file. This commit replaces both the dummy build (which produced $(Build.ArtifactStagingDirectory)/app/example) and the dummy ESRP call (which signed that fake artifact) with a real signing flow against the payload we now build for real. The dummy collect step remains for now and is replaced when the .pkg/.dmg flow lands. Assisted-by: Claude Opus 4.7 Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Run the macos-installer Makefile's `pkg` target to produce .github/macos-installer/disk-image/git-<version>-universal.pkg from the signed payload tree built and ESRP-signed in the preceding steps. Crucially we leave APPLE_INSTALLER_IDENTITY undefined. The Makefile's pkg_cmd has an `ifdef APPLE_INSTALLER_IDENTITY` branch that adds `--sign "<identity>"` to the pkgbuild invocation; with the variable unset, pkgbuild produces an unsigned .pkg, which ESRP then signs in the next commit. This is the whole point of the migration: replace the `productsign`-via-pkgbuild path with ESRP signing of the resulting .pkg. GITHUB_WORKSPACE and VERSION are exported for the same reason as in the payload step: the Makefile reads them directly and there is no GitHub Actions runtime under ADO to set them automatically. Assisted-by: Claude Opus 4.7 Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Sign the unsigned .pkg produced in the previous step. The same ESRP path used for the Mach-O binaries works here too: KeyCode CP-401337-Apple covers both Developer ID Application and Developer ID Installer certs in this account, so MacAppDeveloperSign on a .pkg is the productsign equivalent. git-credential-manager/.azure-pipelines/release.yml uses exactly this pattern (same KeyCode and OperationCode, just pointed at the .pkg instead of the binary tree), so the sign-then-extract cycle through the existing esrp/sign.yml template applies unchanged. folderPath is the macos-installer's disk-image/ directory, where the previous step deposited the unsigned .pkg; the template's ExtractFiles@1 writes the signed .pkg back over it. This replaces the productsign --sign branch in the macos-installer Makefile's pkg_cmd, which we deliberately did not exercise (we left APPLE_INSTALLER_IDENTITY undefined in the previous commit so pkgbuild produced an unsigned .pkg). Assisted-by: Claude Opus 4.7 Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Submit the signed .pkg through ESRP for Apple notarization. Same KeyCode (CP-401337-Apple) and same template as the previous two ESRP calls, but a different OperationCode (MacAppNotarize) and a required BundleId parameter. The bundle identifier comes straight from the macos-installer Makefile's pkg_cmd, which invokes `pkgbuild --identifier com.git.pkg`. Reusing it here keeps the notarization request consistent with the actual identifier baked into the package. This replaces the `xcrun notarytool submit ... --wait` plus `xcrun stapler staple` flow that .github/scripts/notarize.sh performs in the GitHub workflow's `make notarize` target. The ESRP MacAppNotarize operation handles both the submission and the ticket stapling, returning the notarized .pkg back into disk-image/ via the same zip-extract template path the previous sign step used. git-credential-manager/.azure-pipelines/release.yml uses the same operation with its own BundleId; that pipeline is the working reference for this combination of KeyCode, OperationCode, and useArchive: true. Assisted-by: Claude Opus 4.7 Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Run the macos-installer Makefile's `image` target to build .github/macos-installer/git-<version>-universal.dmg from the contents of disk-image/, which by this point contains the signed and notarized .pkg from the previous ESRP step. hdiutil's behaviour is unchanged from the GitHub workflow path; only the trigger is. The final stage step replaces the placeholder dummy collect step that pointed at the long-dead $(Build.ArtifactStagingDirectory)/app directory. The .dmg lands at the macos-installer root and the signed-and-notarized .pkg lives in disk-image/; we move both (named precisely, not globbed) into $(Build.ArtifactStagingDirectory)/_final/, which the job's templateContext.outputs already publishes as the macos_universal pipeline artifact. Naming the files instead of globbing turns "the build skipped one" into a missing-file error rather than a silent half-empty upload, matching the same defensive choice the Linux stage step makes. GITHUB_WORKSPACE and VERSION are exported for the same reason as the earlier macos-installer Makefile invocations: the Makefile reads them directly and ADO does not set them automatically. Assisted-by: Claude Opus 4.7 Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
GitHub Actions has the git-for-windows/setup-git-for-windows-sdk@v1 action that drops a full SDK onto the runner; ADO has no equivalent task, so the Windows job has to bootstrap the SDK by hand before it can run any of the bash-driven build steps the GitHub workflow relies on. The SDK is published as a build-installers.tar.zst snapshot in the `ci-artifacts` rolling release of git-for-windows/git-sdk-64 (and git-sdk-arm64 for ARM64). Add a `sdk_repo` field to each windows_matrix entry so the new download step can pick the right one, and surface it as a job-level variable. Decompressing zstd archives is the awkward bit: Windows 2022's built-in tar.exe (BSD tar) does not understand zstd; Git for Windows' GNU tar can but only if zstd.exe is on the PATH; and we cannot assume Git for Windows is even installed on the agent. The robust path, as suggested by Johannes Schindelin in the planning discussion for this PR, is to download tar.exe, zstd.exe, and msys-zstd-1.dll directly from https://github.com/git-for-windows/git-sdk-64/raw/HEAD/usr/bin into a tools dir, prepend that dir to PATH (in this task via $env:PATH so the extraction can use them, and via ##vso[task.prependpath] so subsequent tasks see them too), then extract the SDK archive with `tar --use-compress-program='zstd -d'` into C:\sdk. The downloaded MSYS2 binaries depend on msys-2.0.dll, which is satisfied either by the agent's bundled MinGit (which the prior setup-git-bash.cmd step puts on the PATH) or, after extraction, by the SDK's own usr/bin. Both paths are kept on PATH after the bootstrap so subsequent bash tasks find a working bash, and the matching MinGW toolchain (mingw64 for x64, clangarm64 for ARM64) is also surfaced. If 1ES network egress later turns out to forbid those raw and release downloads, the same bytes can be vendored via Secure Files or fetched via a partial clone of git-sdk-64; that switch is a follow-up. Assisted-by: Claude Opus 4.7 Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Git for Windows' build tooling (please.sh, signtool.sh, the installer .iss templates, and the MINGW-packages helpers) lives in git-for-windows/build-extra rather than in the SDK snapshot. The GitHub workflow's Windows job clones it into /usr/src/build-extra of the SDK before invoking please.sh; mirror that here. A partial clone (--filter=blob:none) plus --single-branch -b main is enough for everything please.sh needs and avoids pulling the full blob history; it matches the workflow's invocation byte for byte. The bash task picks up the SDK's bash from the PATH set up by the preceding bootstrap step, so /usr/src resolves into the SDK at C:\sdk\usr\src as expected. Assisted-by: Claude Opus 4.7 Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Drive please.sh build-mingw-w64-git from a Bash@3 task using the SDK's bash that the bootstrap step put on the PATH. Outputs land in $(Build.SourcesDirectory)/artifacts/ so the subsequent installer-build step can pass them to please.sh make_installers_from_mingw_w64_git via --pkg= flags. Three small adaptations from the GitHub workflow's source step: The /usr/bin/git trampoline that delegates to the matching MinGW-built git.exe is the same one the workflow writes by hand; makepkg-mingw shells out to plain `git`, and the SDK bash's git candidates would otherwise come from MinGit, not the toolchain we are building against. The user.name / user.email / PACKAGER values used to be the GitHub actor; on ADO there is no equivalent identity, so the initial port hardcodes a build-bot identity. If this needs to attribute to a specific human or service principal later, that is a one-line change here. please.sh's --only-<arch> flag takes the bare CPU name (x86_64 or aarch64), not the toolchain triple. windows_x64 already happens to match (toolchain=x86_64), but windows_arm64's toolchain is clang-aarch64, so a small case statement maps it to aarch64. Adding another arch later would extend the case rather than touch the matrix shape. Two pieces of the workflow's pkg-build step are intentionally not ported in this commit and noted in a comment for follow-up: the per-tarball GPG signing (replaceable with an ESRP PGP operation analogous to the Linux LinuxSign flow if needed downstream), and the MINGW-packages bundle creation, which depends on a PKGBUILD.<tag_name> snapshot that microsoft/git does not currently ship. Assisted-by: Claude Opus 4.7 Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Drive please.sh make_installers_from_mingw_w64_git for both installer and portable variants from a single bash task. The GitHub workflow runs these as separate matrix jobs (one per type/arch combination); the AzP version keeps both builds in the same job so the .pkg.tar.* artifacts produced in the previous step are available without an inter-job artifact passing trip. The PDB archive copy into build-extra/cached-source-packages is the same prerequisite that --include-pdbs needs in the GitHub workflow. Build-extra reads from there to embed PDBs into the installer; without the copy --include-pdbs is silently ineffective. The --pkg= filter that strips signatures and the optional archimport / cvs / p4 / gitweb / doc-man pieces matches the workflow's sed exactly so the resulting .exe sizes are comparable. The two .exe artifacts here are still unsigned: the GitHub workflow relies on Inno Setup invoking signtool.sh via a git signtool alias to sign the installer at build time, and a post-build git signtool to sign the portable. Neither path is wired here; the next commit applies ESRP signing to both .exe artifacts after the build, which is the migration target this PR series is moving towards. The SHA-256 sidecar the workflow emits follows in that same next commit, since its content depends on the signed bytes. Assisted-by: Claude Opus 4.7 Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Replace the dummy ESRP-signing block (which copied calc.exe into example*.exe and signed those) and the dummy collect block (which xcopy'd the same fake artifacts into _final/) with the real post-build flow. The signing call hands artifacts/Git-*.exe and artifacts/PortableGit-*.exe to the existing .azure-pipelines/esrp/windows/esrpsign.sh wrapper that mjcheetham landed earlier. That script reads ESRP_TOOL and ESRP_AUTH from the env block (set by the setup.yml template above) and SYSTEM_ACCESSTOKEN from $(System.AccessToken). The wrapper script is the same one that a future `git signtool` alias is intended to delegate to per the comment on commit 406d385, so signing the installer at Inno Setup time becomes a follow-up that does not require touching this step. The SHA-256 sidecar moves here from the build commit so the hashes reflect the bytes that ship: ESRP signing rewrites the .exe contents, so a SHA-256 computed before signing would mismatch the released artifact. When ESRP is disabled the openssl call still runs against the unsigned binaries, which is the right answer in that case. The final cp into $(Build.ArtifactStagingDirectory)/_final/ ships exactly Git-*.exe, PortableGit-*.exe, and sha-256.txt; the templateContext.outputs.pipelineArtifact already publishes that directory as the windows_x64 / windows_arm64 artifact. Assisted-by: Claude Opus 4.7 Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Port the
build-git-installersGitHub workflow at.github/workflows/build-git-installers.ymlto the Azure Pipeline scaffolded onmicrosoft/azpat.azure-pipelines/release.yml, replacing the workflow'sdebsigs/ Apple Developer cert /git signtoolsigning flows with ESRP throughout. All three OS jobs (Linux, macOS, Windows) are covered; each commit is small and reviewable.The macOS pattern follows what
git-credential-manager/.azure-pipelines/release.ymlalready does (sameCP-401337-AppleKeyCode forMacAppDeveloperSigncovering both binary codesign andproductsign-equivalent .pkg signing, thenMacAppNotarizewith the bundle ID baked into.github/macos-installer/Makefile, all wrapped in the existinguseArchive: truezip-then-sign-then-extract template).The Windows job bootstraps the Git for Windows SDK manually because there is no AzP equivalent of
git-for-windows/setup-git-for-windows-sdk@v1: downloadtar.exe/zstd.exe/msys-zstd-1.dllfromgit-for-windows/git-sdk-64, fetch the matchingbuild-installers.tar.zstsnapshot from the SDK repo'sci-artifactsrolling release (git-sdk-64for x64,git-sdk-arm64for ARM64), extract toC:\sdk, and reuse the existing.azure-pipelines/esrp/windows/esrpsign.shwrapper for post-build signing.The Linux job's first run (1ES build IDs 141 and 154) showed that the
GitClientPME-1ESHostedPool-{intel,arm64}-pcimages silently ignore thecontainer:job-level directive; the first commit therefore drops the workflow's container path entirely and logslsb_release -ainstead, withsudo apt-getfor the package install.Per-commit messages carry the full implementation reasoning (which Make flags, why each ESRP operation, what the GitHub workflow's container/env-var/Node.js workarounds correspond to here).
A few items deliberately deferred as follow-ups:
MINGW-packages.bundlecreation (depends on aPKGBUILD.<tag_name>snapshot that microsoft/git does not currently ship).git signtoolalias (mjcheetham's stated next step in 406d385 (azure-pipelines: add ESRP code signing, 2026-04-30)).debsigs --verify --checkafter signing).linux_*/.tar.gzglob (we do not yet produce a tarball alongside the .deb).Marked as draft because the pipeline has only been triggered for the Linux jobs so far (and that one trip is what produced the container/sudo discovery above); the macOS and Windows steps have not been exercised against the 1ES pools yet.
Supersedes #909.