Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Rewrite the entire Azure DevOps build system (#15808)
This pull request rewrites the entire Azure DevOps build system. The guiding principles behind this rewrite are: - No pipeline definitions should contain steps (or tasks) directly. - All jobs should be in template files. - Any set of steps that is reused across multiple jobs must be in template files. - All artifact names can be customized (via a property called `artifactStem` on all templates that produce or consume artifacts). - No compilation happens outside of the "Build" phase, to consolidate the production and indexing of PDBs. - **Building the project produces a `bin` directory.** That `bin` directory is therefore the primary currency of the build. Jobs will either produce or consume `bin` if they want to do anything with the build outputs. - All step and job templates are named with `step` or `job` _first_, which disambiguates them in the templates directory. - Most jobs can be run on different `pool`s, so that we can put expensive jobs on expensive build agents and cheap jobs on cheap build agents. Some jobs handle pool selection on their own, however. Our original build pipelines used the `VSBuild` task _all over the place._ This resulted in Terminal being built in myriad ways, different for every pipeline. There was an attempt at standardization early on, where `ci.yml` consumed jobs and steps templates... but when `release.yml` was added, all of that went out the window. The new pipelines are consistent and focus on a small, well-defined set of jobs: - `job-build-project` - This is the big one! - Takes a list of build configurations and platforms. - Produces an artifact named `build-PLATFORM-CONFIG` for the entire matrix of possibilities. - Optionally signs the output and produces a bill of materials. - Admittedly has a lot going on. - `job-build-package-wpf` - Takes a list of build configurations and platforms. - Consumes the `build-` artifact for every config/platform possibility, plus one for "Any CPU" (hardcoded; this is where the .NET code builds) - Produces one `wpf-nupkg-CONFIG` for each configuration, merging all platforms. - Optionally signs the output and produces a bill of materials. - `job-merge-msix-into-bundle` - Takes a list of build configurations and platforms. - Consumes the `build-` artifact for every config/platform - Produces one `appxbundle-CONFIG` for each configuration, merging all platforms for that config into one `msixbundle`. - Optionally signs the output and produces a bill of materials. - `job-package-conpty` - Takes a list of build configurations and platforms. - Consumes the `build-` artifact for every config/platform - Produces one `conpty-nupkg-CONFIG` for each configuration, merging all platforms. - Optionally signs the output and produces a bill of materials. - `job-test-project` - Takes **one** build config and **one** platform. - Consumes `build-PLATFORM-CONFIG` - Selects its own pools (hardcoded) because it knows about architectures and must choose the right agent arch. - Runs tests (directly on the build agent). - `job-run-pgo-tests` - Just like the above, but runs tests where `IsPgo` is `true` - Collects all of the PGO counts and publishes a `pgc-intermediates` artifact for that platform and configuration. - `job-pgo-merge-pgd` - Takes **one** build config and multiple platforms. - Consumes `build-$platform-CONFIG` for each platform. - Consumes `pgc-intermediates-$platform-CONFIG` for each platform. - Merges the `pgc` files into `pgd` files - Produces a new `pgd-` artifact. - `job-pgo-build-nuget-and-publish` - Consumes the `pgd-` artifact from above. - Packs it into a `nupkg` and publishes it. - `job-submit-windows-vpack` - Only expected to run against `Release`. - Consumes the `appxbundle-CONFIG` artifact. - Publishes it to a vpack for Windows to consume. - `job-check-code-format` - Does not use artifacts. Runs `clang-format`. - `job-index-github-codenav` - Does not use artifacts. Fuzz submission is broken due to changes in the `onefuzz` client. I have removed the compliance and security build because it is no longer supported. Finally, this pull request has some additional benefits: - I've expanded the PGO build phase to cover ARM64! - We can remove everything Helix-related except the WTT parser - We no longer depend on Helix submission or Helix pools - The WPF control's inner DLLs are now codesigned (#15404) - Symbols for the WPF control, both .NET and C++, are published alongside all other symbols. - The files we submit to ESRP for signing are batched up into a single step[^1] Closes #11874 Closes #11974 Closes #15404 [^1]: This will have to change if we want to sign the individual per-architecture `.appx` files before bundling so that they can be directly installed. (cherry picked from commit 69eff7e) Service-Card-Id: 90183388 Service-Version: 1.18
- Loading branch information
bb64c79
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@check-spelling-bot Report
🔴 Please review
See the 📜action log for details.
Unrecognized words (1)
LASTEXITCODE
Previously acknowledged words that are now absent
ACCESSTOKEN BUILDURI COLLECTIONURI countof dhandler dmp etcoreapp fileurl HKLM homeglyphs IXMP lastexitcode listproperties logissue minimizeall preinstalled procs SOURCEBRANCH suiteless TEAMPROJECT testbuildplatform testdlls testmode testnameprefix testtimeout untests xxyyzz :arrow_right:To accept ✔️ these unrecognized words as correct and remove the previously acknowledged and now absent words, run the following commands
... in a clone of the git@github.com:microsoft/terminal.git repository
on the
release-1.18
branch (ℹ️ how do I use this?):Pattern suggestions ✂️ (1)
You could add these patterns to .github/actions/spelling/patterns/bb64c797ab8eb493e8ec63d2b6ce018fbf606d94.txt:
Warnings (1)
See the 📜action log for details.
See ℹ️ Event descriptions for more information.
✏️ Contributor please read this
By default the command suggestion will generate a file named based on your commit. That's generally ok as long as you add the file to your commit. Someone can reorganize it later.
expect.txt
and theexcludes.txt
files.If the listed items are:
.github/actions/spelling/allow/names.txt
..github/actions/spelling/allow/
..github/actions/spelling/expect/
..github/actions/spelling/patterns/
.See the
README.md
in each directory for more information.🔬 You can test your commits without appending to a PR by creating a new branch with that extra change and pushing it to your fork. The check-spelling action will run in response to your push -- it doesn't require an open pull request. By using such a branch, you can limit the number of typos your peers see you make. 😉
If the flagged items are 🤯 false positives
If items relate to a ...
binary file (or some other file you wouldn't want to check at all).
Please add a file path to the
excludes.txt
file matching the containing file.File paths are Perl 5 Regular Expressions - you can test yours before committing to verify it will match your files.
^
refers to the file's path from the root of the repository, so^README\.md$
would exclude README.md (on whichever branch you're using).well-formed pattern.
If you can write a pattern that would match it,
try adding it to the
patterns.txt
file.Patterns are Perl 5 Regular Expressions - you can test yours before committing to verify it will match your lines.
Note that patterns can't match multiline strings.