[Scenario] DXC Engineering System Debt

[damyanp]

The engineering system around DXC's official builds has become unwieldly due to the shifting landscape around it. Most, if not all, of DXC is now open source and so there is no real requirement to maintain internal repos and build pipelines. Many DXC developers prefer to use ninja rather than msbuild, and many DXC developers need to build in non-Windows environments.

Keeping this all working well is expensive - for example, it often takes weeks to spin up a new release branch when it should take minutes. We should take some time to fix some targeted issues that make the overall DXC build system unnecessarily complicated and fragile.

We should aim to remove the need for the internal "wintools" repo entirely for general DXC releases. Build pipelines should be driven from yml files stored in the git repo.

While doing this, we should make it so that any packages we build (vpacks, nugets etc.) contain a breadcrumb indicating the SHA that they were built from.

[MarijnS95]

That sounds neat, and it'd be great to bring back per-commit artifacts from when appveyor was still around.

Also linking in #6184 / #7174 where we're observing SPIRV-Tools validation errors on the released aarch64 Windows libraries, which we cannot reproduce if we compile DXC from source ourselves. Without insight in the build pipelines used upstream, it's nigh impossible to help debug the cause.

[MarijnS95]

For reference we've been rolling our own GitHub-Actions-based infrastructure to produce such artifacts for multiple targets/architectures, it's incredibly easy to set up:

https://github.com/Traverse-Research/dxc-build/blob/main/.github/workflows/build-dxc.yaml

[damyanp]

Thanks - this is out of scope for this scenario though. If there's not an existing issue requesting per-commit build artifacts then please feel free to create one.

[MarijnS95]

@damyanp this is not just a request to provide per-commit build artifacts, since such artifacts should be required in the first place to build-test the repository and subsequently run tests on, right?

[damyanp]

Part of the issue here is with retention of the artifacts. The actual build pipelines that we use to build official, signed, releases are quite locked down and so we wouldn't be able to provide general access to those. I think possibly a takeaway here is that I need to tighten up the language in the description. This is about "official" builds rather than pull request builds.