Skip to content

Add NativeAOT aggregate executable shims#128553

Draft
Copilot wants to merge 11 commits into
mainfrom
copilot/extend-fat-executable-to-linux-macos
Draft

Add NativeAOT aggregate executable shims#128553
Copilot wants to merge 11 commits into
mainfrom
copilot/extend-fat-executable-to-linux-macos

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented May 25, 2026

Enable NativeAOT shared library projects to aggregate multiple executable entry points into one native library and emit small per-app shim executables.

This addresses scenarios where shipping multiple NativeAOT executables as a bundle would otherwise duplicate megabytes of code and data across each executable. A shared NativeAOT library can now mark executable ProjectReference items with IncludeInAggregateExecutable="true". ILC exports a distinct startup entry point for each referenced executable, and MSBuild generates platform-specific shims that dispatch to the appropriate exported entry point.

The aggregate executable mode is inferred automatically when NativeLib=Shared and one or more project references are marked with IncludeInAggregateExecutable="true".

  • Compiler plumbing

    • Adds --aggregateexe:assemblyName=MainEntryPointName handling in ILC.
    • Roots per-assembly startup stubs with distinct exported entry point symbols.
  • Build integration

    • Generates platform-specific shim executables for aggregate executable project references.
    • Supports Windows, Linux, and macOS shim generation.
    • Preserves per-reference linker subsystem selection for Windows shims.
    • Sets same-directory runtime lookup for generated Unix shims.
    • Links ELF shims against the adjacent aggregate shared library by library name instead of embedding a build-machine absolute path.
  • Regression coverage

    • Adds a NativeAOT aggregate executable test with two shim executables dispatching into one shared native library.
    • Enables the test across Windows, Linux, and macOS.
    • Emits Helix extra-executable metadata for the dependent Unix test shim so execute permissions are restored after payload copy.

Example project shape:

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <OutputType>Library</OutputType>
    <NativeLib>Shared</NativeLib>
  </PropertyGroup>

  <ItemGroup>
    <ProjectReference Include="HelloExe\HelloExe.csproj" IncludeInAggregateExecutable="true" />
    <ProjectReference Include="ValidatorExe\ValidatorExe.csproj" IncludeInAggregateExecutable="true" />
  </ItemGroup>
</Project>

Copilot AI and others added 6 commits May 25, 2026 09:47
Agent-Logs-Url: https://github.com/dotnet/runtime/sessions/69bd1576-bcad-4ee1-aa5e-af9513609eea

Co-authored-by: MichalStrehovsky <13110571+MichalStrehovsky@users.noreply.github.com>
Agent-Logs-Url: https://github.com/dotnet/runtime/sessions/69bd1576-bcad-4ee1-aa5e-af9513609eea

Co-authored-by: MichalStrehovsky <13110571+MichalStrehovsky@users.noreply.github.com>
Agent-Logs-Url: https://github.com/dotnet/runtime/sessions/69bd1576-bcad-4ee1-aa5e-af9513609eea

Co-authored-by: MichalStrehovsky <13110571+MichalStrehovsky@users.noreply.github.com>
Agent-Logs-Url: https://github.com/dotnet/runtime/sessions/69bd1576-bcad-4ee1-aa5e-af9513609eea

Co-authored-by: MichalStrehovsky <13110571+MichalStrehovsky@users.noreply.github.com>
Agent-Logs-Url: https://github.com/dotnet/runtime/sessions/69bd1576-bcad-4ee1-aa5e-af9513609eea

Co-authored-by: MichalStrehovsky <13110571+MichalStrehovsky@users.noreply.github.com>
Agent-Logs-Url: https://github.com/dotnet/runtime/sessions/69bd1576-bcad-4ee1-aa5e-af9513609eea

Co-authored-by: MichalStrehovsky <13110571+MichalStrehovsky@users.noreply.github.com>
Copilot AI review requested due to automatic review settings May 25, 2026 11:51
Copilot AI review requested due to automatic review settings May 25, 2026 11:51
@MichalStrehovsky MichalStrehovsky changed the title Extend NativeAOT fat executables to Unix Implement "fat" executables May 25, 2026
@MichalStrehovsky MichalStrehovsky changed the title Implement "fat" executables Add NativeAOT fat executable shims May 25, 2026
@dotnet-policy-service
Copy link
Copy Markdown
Contributor

Tagging subscribers to this area: @agocke, @dotnet/ilc-contrib
See info in area-owners.md if you want to be subscribed.

@jkotas
Copy link
Copy Markdown
Member

jkotas commented May 25, 2026

fat executable" (I don't like the name, but I don't have better names)

Yeah, fat executables are a thing https://www.bing.com/search?q=fat+executables . I had to re-read the PR description a few times to figure out that this is completely unrelated. Some naming suggestions "aggregate executable", "publish as aggregate", "multi app executable".

On Unix, it is not unusual to bundle multiple tools into a single binary. It is typically achieved by creating a single binary and linking the single binary to different file names. For example, https://busybox.net/ uses this technique. This PR is trying to provide equivalent characteristics by working around lack of first-class support for links in NuGet and other .NET tooling. Does this sound accurate?

We have similar asks for non-NativeAOT too #53834 . I am wondering whether we should think about unified UX for this that can work for different form factors.

@agocke
Copy link
Copy Markdown
Member

agocke commented May 25, 2026

On Unix, it is not unusual to bundle multiple tools into a single binary. It is typically achieved by creating a single binary and linking the single binary to different file names.

Yup, I believe this is called a "polymorphic executable" in Unix-land. It does seem like a useful pattern. The symlinks are very nice because you don't even pay a few KB per exe, it's more like a few hundred bytes. And when you're busybox with 100+ binaries, the difference is meaningful

@agocke
Copy link
Copy Markdown
Member

agocke commented May 25, 2026

One note, since I hit this when trying to support symlinks for the muxer on windows: the windows dynamic loader always resolves from the link directory. So a hard link/symlink solution for Windows would be hard to support if the app has dynamic dependencies.

@MichalStrehovsky
Copy link
Copy Markdown
Member

The scheme added here is something that can be supported crossplat.

The symlinks don't work crossplat, one cannot ZIP them up etc. The symlink scheme and lack of shims is the exact reason why LLVM is a 700 MB download on Windows: the distribution ships clang.exe, clang++.exe, clang-cl.exe, clang-cpp.exe, each a 100 MB binary that is 100% byte identical to the other. (These are not the only files with the problem.)

Without a shim, we'll not be able to set per-EXE Win32 resources and the best we can do for places that surface it (like Task Manager on Windows) is to call these ".NET Host":

image

@MichalStrehovsky
Copy link
Copy Markdown
Member

Another reason why symlinks are a poor choice on Windows is that on Windows we have /subsystem:console and /subsystem:gui and it's quite common to have a GUI app alongside some CLI apps. I specifically asked Copilot to support this when it was doing the MSBuild changes, so one can specify the subsytem on the ProjectReference and the shim will be the correct EXE type.

@jkotas
Copy link
Copy Markdown
Member

jkotas commented May 26, 2026

I agree that executable links/symlinks do not work well on Windows and in .NET in general. I am ok with building it without links/symlinks.

I think this needs:

  • Less confusing name
  • Non-NativeAOT consideration: Can this UX work for non-NativeAOT? What would it take to enable it?

@MichalStrehovsky
Copy link
Copy Markdown
Member

Non-NativeAOT consideration: Can this UX work for non-NativeAOT? What would it take to enable it?

I did not think about this as a publicly documented feature. We have immediate need for this in .NET 11 since the goals to shrink the SDK download and making the SDK more responsive are at odds with each other.

For native AOT, extending the "publish a native library" concept works naturally since the native library is the thing we're going to publish. It just adds the exports on top of whatever exports the user already has, and creates the shims. Feature switches, optimization settings, etc. will be set at this level. The experience doesn't exactly match the PublishTrimmed and PublishAot promises (the native library is not runnable without a publish and the settings the library is published with may not match the settings the individual exe projects have when the user debugs them).

I think we'd need a new .bundeproj project type or something like that with first-class SDK support to have something that generalizes to non-publish AOT experiences (and makes the experience consistent for native AOT too - e.g. forces the individual EXEs to have the same feature switch settings).

I'll make copilot gate this behind EnablePreviewFeatures property. Then we're free to break it in a couple years from now when we add .bundleproj.

Copy link
Copy Markdown
Member

@MichalStrehovsky MichalStrehovsky left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment thread src/coreclr/nativeaot/BuildIntegration/Microsoft.NETCore.Native.Publish.targets Outdated
Comment thread src/coreclr/nativeaot/BuildIntegration/Microsoft.NETCore.Native.Publish.targets Outdated
Comment thread src/tests/nativeaot/FatExecutableLibrary/FatLibrary.csproj Outdated
Comment thread src/coreclr/nativeaot/BuildIntegration/Microsoft.NETCore.Native.Unix.targets Outdated
Comment thread src/coreclr/nativeaot/BuildIntegration/Microsoft.NETCore.Native.Unix.targets Outdated
Agent-Logs-Url: https://github.com/dotnet/runtime/sessions/f0f54e39-2f6a-48af-88ed-f5724228b103

Co-authored-by: MichalStrehovsky <13110571+MichalStrehovsky@users.noreply.github.com>
Copilot AI requested review from Copilot and removed request for Copilot May 26, 2026 05:57
Agent-Logs-Url: https://github.com/dotnet/runtime/sessions/f0f54e39-2f6a-48af-88ed-f5724228b103

Co-authored-by: MichalStrehovsky <13110571+MichalStrehovsky@users.noreply.github.com>
Copilot AI requested review from Copilot and removed request for Copilot May 26, 2026 06:02
Agent-Logs-Url: https://github.com/dotnet/runtime/sessions/f0f54e39-2f6a-48af-88ed-f5724228b103

Co-authored-by: MichalStrehovsky <13110571+MichalStrehovsky@users.noreply.github.com>
Copilot AI requested review from Copilot and removed request for Copilot May 26, 2026 06:06
Copy link
Copy Markdown
Contributor Author

Copilot AI commented May 26, 2026

@copilot

Addressed the aggregate executable rename, automatic ProjectReference-based enablement, test runner cleanup, and required NativeAOT validation in 93f93d0.

Copilot AI changed the title Add NativeAOT fat executable shims Add NativeAOT aggregate executable shims May 26, 2026
Copilot AI requested a review from MichalStrehovsky May 26, 2026 06:08
@MichalStrehovsky
Copy link
Copy Markdown
Member

@copilot the test still doesn't work on non-Windows:

06:41:06.525 Running test: nativeaot/AggregateExecutableLibrary/AggregateLibrary/AggregateLibrary.cmd
/datadisks/disk1/work/A95D094F/w/96CE0873/e/nativeaot/AggregateExecutableLibrary/AggregateLibrary//native/ValidatorExe: error while loading shared libraries: /__w/1/s/artifacts/tests/coreclr/linux.x64.Debug/nativeaot/AggregateExecutableLibrary/AggregateLibrary/native/libAggregateLibrary.so: cannot open shared object file: No such file or directory

Return code:      1
Raw output file:      /datadisks/disk1/work/A95D094F/w/96CE0873/uploads/AggregateExecutableLibrary/AggregateLibrary/output.txt
Raw output:
BEGIN EXECUTION
/datadisks/disk1/work/A95D094F/p/nativeaottest.sh /datadisks/disk1/work/A95D094F/w/96CE0873/e/nativeaot/AggregateExecutableLibrary/AggregateLibrary/ ValidatorExe.dll ''
Expected: 100
Actual: 127
END EXECUTION - FAILED

Looks like we're embedding the full file path to the .so library on the build machine (that doesn't exist on the test machine but the library is next to the executable by convention).

Agent-Logs-Url: https://github.com/dotnet/runtime/sessions/ff586a1f-b533-4271-a092-8a030ea016ec

Co-authored-by: MichalStrehovsky <13110571+MichalStrehovsky@users.noreply.github.com>
Copilot AI requested review from Copilot and removed request for Copilot May 26, 2026 08:10
Copy link
Copy Markdown
Contributor Author

Copilot AI commented May 26, 2026

@copilot the test still doesn't work on non-Windows:

...

Fixed in 9cea2e3. The Unix shim now links the aggregate library by search path/name instead of passing the absolute build-machine path; local validation passed and readelf shows NEEDED libAggregateLibrary.so with RUNPATH $ORIGIN.

@am11
Copy link
Copy Markdown
Member

am11 commented May 26, 2026

On Unix, even shim executable isn't necessary. If we want an extra-diet mode, we can set chmod +x on shared aggregate lib and conditionally disable:

<LinkerArg Include="-Wl,-e,0x0" Condition="'$(NativeLib)' == 'Shared' and '$(_IsApplePlatform)' != 'true'" />

to enable ./libfoo.so --arg. e.g. one of the most famous one is:

$ ./libc.so.6 --help
GNU C Library (Ubuntu GLIBC 2.43-2ubuntu2) stable release version 2.43.
Copyright (C) 2026 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.
Compiled by GNU CC version 15.2.0.
libc ABIs: UNIQUE ABSOLUTE
Minimum supported kernel: 3.7.0
For bug reporting instructions, please see:
<https://bugs.launchpad.net/ubuntu/+source/glibc/+bugs>.

@MichalStrehovsky
Copy link
Copy Markdown
Member

On Unix, even shim executable isn't necessary. If we want an extra-diet mode, we can set chmod +x on shared aggregate lib and conditionally disable:

You mean to save one shim executable? I'm not sure if it's worth complicating things to save ~5 kB.

@MichalStrehovsky
Copy link
Copy Markdown
Member

@copilot still doesn't work on non-Windows. Should something chmod +x HelloExe?

09:07:49.251 Running test: nativeaot/AggregateExecutableLibrary/AggregateLibrary/AggregateLibrary.cmd
Unhandled exception. System.ComponentModel.Win32Exception (13): An error occurred trying to start process '/datadisks/disk1/work/BA7B0A07/w/B10E09DE/e/nativeaot/AggregateExecutableLibrary/AggregateLibrary/native/HelloExe' with working directory '/datadisks/disk1/work/BA7B0A07/w/B10E09DE/e/nativeaot/AggregateExecutableLibrary/AggregateLibrary'. Permission denied
   at Microsoft.Win32.SafeHandles.SafeProcessHandle.ForkAndExecProcess(ProcessStartInfo, String, String[], IDictionary`2, String, Boolean, UInt32, UInt32, UInt32[], SafeFileHandle, SafeFileHandle, SafeFileHandle, Boolean, SafeHandle[], ProcessWaitState.Holder&, Boolean)
   at Microsoft.Win32.SafeHandles.SafeProcessHandle.StartCore(ProcessStartInfo, SafeFileHandle, SafeFileHandle, SafeFileHandle, SafeHandle[], ProcessWaitState.Holder&)
   at System.Diagnostics.Process.StartCore(ProcessStartInfo, SafeFileHandle, SafeFileHandle, SafeFileHandle, SafeHandle[])
   at System.Diagnostics.Process.Start()
   at System.Diagnostics.Process.Start(ProcessStartInfo)
   at Program.<<Main>$>d__0.MoveNext()
--- End of stack trace from previous location ---
   at Program.<Main>(String[] args)
DOTNET_DbgEnableMiniDump is set and the createdump binary does not exist: /datadisks/disk1/work/BA7B0A07/w/B10E09DE/e/nativeaot/AggregateExecutableLibrary/AggregateLibrary/native/createdump
/datadisks/disk1/work/BA7B0A07/p/nativeaottest.sh: line 15: 11028 Aborted                 (core dumped) $_DebuggerFullPath $1/native/$exename "${@:3}"

Return code:      1
Raw output file:      /datadisks/disk1/work/BA7B0A07/w/B10E09DE/uploads/AggregateExecutableLibrary/AggregateLibrary/output.txt
Raw output:
BEGIN EXECUTION
/datadisks/disk1/work/BA7B0A07/p/nativeaottest.sh /datadisks/disk1/work/BA7B0A07/w/B10E09DE/e/nativeaot/AggregateExecutableLibrary/AggregateLibrary/ ValidatorExe.dll ''
Expected: 100
Actual: 134
END EXECUTION - FAILED

Agent-Logs-Url: https://github.com/dotnet/runtime/sessions/5b3bca38-fa3b-46a4-840c-50b496db204a

Co-authored-by: MichalStrehovsky <13110571+MichalStrehovsky@users.noreply.github.com>
Copilot AI requested review from Copilot and removed request for Copilot May 26, 2026 10:35
Copy link
Copy Markdown
Contributor Author

Copilot AI commented May 26, 2026

@copilot still doesn't work on non-Windows. Should something chmod +x HelloExe?

...

Fixed in d273bad. The test now emits helix-extra-executables.list for native/HelloExe, so Helix reapplies execute permission after copying the payload; full NativeAOT validation passed locally.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants