Share common native infrastructure

[ericstj]

• This issue is blocking
• This issue is causing unreasonable pain

Today dotnet repos that build cross-platform native code all define their own settings for CMake. This can lead to differences like different optimizations / symbols / shared libraries etc. At the very least it's a ton of duplication and redundant work.

We should try to find how much of the dotnet/runtime native CMake infrastructure is shareable and put that in arcade. Then we can have repositories use that infrastructure rather than their locally defined scripts. One candidate we have for this is dotnet/machinelearning, but I bet there are others.

cc @mmitche @jkotas @agocke

[mmitche]

/cc @markwilkie

[michellemcdaniel]

/cc @Chrisboh

[Chrisboh]

@ericstj we won't have context into each and every build to know what could / could not be shared. Do you happen to have a list of things that should be shared and made into packages?

[ericstj]

The sort of things I was noticing was compiler and linker flags as well as properties and conditions around those. I would hope that we had a common recipe for producing native binaries so that repos don't get things wrong. The sorts of problems I've seen are different symbols flags, missing optimizations or security flags, inconsistent versioning, and arbitrary differences in platform representation.

For managed code Arcade solves most of these problems by setting up good defaults in MSBuild so that a project needs very little boilerplate. For native code there is no help, and repos need to set everything up on their own.

Success here would look like adding a "recipe" for a native binary to the arcade template with minimal boilerplate in the repo and ensuring that binary satisfies all release requirements.

[Chrisboh]

Do you feel this would be something that could be sorted out in the VMR work and once we get those pieces unified we can start pulling things out into arcade that make sense?

[jkotas]

VMR is not going to make this happen as a side-effect, nor make it significantly easier to do.

The shared native infrastructure lives under https://github.com/dotnet/arcade/tree/main/eng/common/native .

This work is about carving out more project-neutral native build support (e.g. carving out parts from https://github.com/dotnet/runtime/tree/main/eng/native), moving them under https://github.com/dotnet/arcade/tree/main/eng/common/native, and switching repos that have native build to use it. It is open-ended problem.

I would say that this work would be best done by maintainers of repos that have native build and want to reduce the cost of maintenance of their repo-local native build infrastructure.

[markwilkie]

Is there an effort currently underway that this issue might fit into?