NativeAOT build as StaticLib crashes on iOS when link together with other static lib

[Magicqy]

Description

dotnet native aot build as static library does not work

call function export from native aot will crash on osx-arm64 and ios-arm64

Reproduction Steps

this is an csproj build static library for osx-arm64, and try to link and call fucntion from an simple C program

running result: segmentation fault

test-static.zip

reproduce steps:

run from bash: run-static.sh <net9|net10>

just a simple function to export

using System;
using System.Runtime.CompilerServices;
using System.Runtime.InteropServices;

public static class Program
{
[UnmanagedCallersOnly(EntryPoint = nameof(TestStaticFunc), CallConvs = new Type[] { typeof(CallConvCdecl) })]
public static void TestStaticFunc()
{
}
}

call from C program

#include <stdio.h>

extern void TestStaticFunc();

int main() {
printf("test static start...\n");
TestStaticFunc();
printf("test static successfully.\n");
return 0;
}

Expected behavior

expected running output:
test static start...
test static successfully.

Actual behavior

segmentation fault, program crash after outpu: test static start...

Regression?

No response

Known Workarounds

dynamic libraray works well
I make dynamic libraray runs correctly on win-x64 (.dll) osx-arm64 (.dylib) and android (.so)
but AppStore reject my dylib build for ios-arm64, so I have to build it as static lib, but it crash in my ios app

Configuration

my computer is mac book with Apple M1 Pro chip
my ios device is iphone 12 with iOS 16.3.1
VSCode is 1.101.0 (Universal)
dotnet SDK installed is 9.0.6 and 10.0.0-preview.5.25277.114

dotnet --info
.NET SDK:
Version: 10.0.100-preview.5.25277.114
Commit: ddf39a1b46
Workload version: 10.0.100-manifests.35d2efbf
MSBuild version: 17.15.0-preview-25277-114+ddf39a1b4

Runtime Environment:
OS Name: Mac OS X
OS Version: 14.5
OS Platform: Darwin
RID: osx-arm64
Base Path: /usr/local/share/dotnet/sdk/10.0.100-preview.5.25277.114/

.NET workloads installed:
There are no installed workloads to display.
Configured to use workload sets when installing new manifests.
Workloads are configured to install and update using workload versions, but none were found. Run "dotnet workload restore" to install a workload version.

Host:
Version: 10.0.0-preview.5.25277.114
Architecture: arm64
Commit: ddf39a1b46

.NET SDKs installed:
9.0.301 [/usr/local/share/dotnet/sdk]
10.0.100-preview.5.25277.114 [/usr/local/share/dotnet/sdk]

.NET runtimes installed:
Microsoft.AspNetCore.App 9.0.6 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 10.0.0-preview.5.25277.114 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
Microsoft.NETCore.App 9.0.6 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
Microsoft.NETCore.App 10.0.0-preview.5.25277.114 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]

Other architectures found:
None

Environment variables:
Not set

global.json file:
Not found

Other information

No response

[dotnet-policy-service]

Tagging subscribers to this area: @agocke, @MichalStrehovsky, @jkotas
See info in area-owners.md if you want to be subscribed.

[agocke]

Hi, could you provide more info about this:

but AppStore reject my dylib build for ios-arm64

[Magicqy]

Hi, could you provide more info about this:

but AppStore reject my dylib build for ios-arm64

first I put dylib build for ios-arm64 in my iOS App, but AppStore reply iOS App can't include dylib outside .framework
then I build a simple .framework with XCode, and put my dylib inside it
then AppStore reply .framework contains disallow file 'Frameworks' (which is an directory)

my .framework is like (Infra.dylib is build from dotnet NativeAOT)

I search for some solution, many people says AppStore only allow dylibs provided by iOS system to be used in iOS App,

so I have to try build my dotnet NatvieAOT lib as StaticLib( .a file ), merge into my iOS App's main executable

but it can't run correctly, crashed when calling functions from C# side (export by UnmanagedCallersOnly)

[Magicqy]

I make this simple repo project to figure out why static lib does not work

in this file (I think it's startup for dotnet NativeAOT env) https://github.com/dotnet/runtime/blob/0d628dae1cb5b8d0eb76d9549134edfe30d6219c/src/coreclr/nativeaot/Bootstrap/main.cpp

simple steps for NativeAOT startup:

InitializeRuntime
call native main function
then call managed_main function

for DLL mode, no main and manged_main function

#ifdef NATIVEAOT_DLL
static struct InitializeRuntimePointerHelper
{
    InitializeRuntimePointerHelper()
    {
        RhSetRuntimeInitializationCallback(&InitializeRuntime);
    }
} initializeRuntimePointerHelper;
#endif // NATIVEAOT_DLL

static struct InitializeRuntimePointerHelper 's contructor calls InitializeRuntime first, then call UnmanagedCallersOnly C# fucntions works

for Exe mode, this bootstrap file has define a native main function
it call InitializeRuntime and then call __managed_Main function, also works well

#if defined(_WIN32)
int CDECL wmain(int argc, wchar_t* argv[])
#else
int main(int argc, char* argv[])
#endif
{
    int initval = InitializeRuntime();
    if (initval != 0)
        return initval;

#if defined(DEBUG) && defined(_WIN32)
    // quick_exit works around Debug UCRT shutdown issues: https://github.com/dotnet/runtime/issues/108640
    quick_exit(__managed__Main(argc, argv));
#else
    return __managed__Main(argc, argv);
#endif
}

for StaticLib mode
InitializeRuntime is not called, so my UnmanagedCallersOnly C# fucntions calls will crash for not inited dotnet runtime
because the main function or InitializeRuntimePointerHelper from dotnet is not included

this InitializeRuntime is not exported as golbal symbol, so I can't call it from my native code

static int InitializeRuntime()
{
    if (!RhInitialize(
#ifdef NATIVEAOT_DLL
        /* isDll */ true
#else
        /* isDll */ false
#endif
        ))
        return -1;

    void * osModule = PalGetModuleHandleFromPointer((void*)&NATIVEAOT_ENTRYPOINT);

    // TODO: pass struct with parameters instead of the large signature of RhRegisterOSModule
    if (!RhRegisterOSModule(
        osModule,
        (void*)&__managedcode_a, (uint32_t)((char *)&__managedcode_z - (char*)&__managedcode_a),
        (void*)&__unbox_a, (uint32_t)((char *)&__unbox_z - (char*)&__unbox_a),
        (void **)&c_classlibFunctions, _countof(c_classlibFunctions)))
    {
        return -1;
    }

    InitializeModules(osModule, __modules_a, (int)((__modules_z - __modules_a)), (void **)&c_classlibFunctions, _countof(c_classlibFunctions));

#ifdef NATIVEAOT_DLL
    // Run startup method immediately for a native library
    __managed__Startup();
#endif // NATIVEAOT_DLL

    return 0;
}

so I think maybe is a bug for StaticLib mode of NativeAOT
my repo project is build dotnet NativeAOT .a for osx-arm64, and using it in an simple console project writen by C, it crash just like iOS App

[Magicqy]

to verify my guess the reason is dotnet runtime not inited in StaticLib mode
I modify my repo project

I link libbootstrapper.o file to my C project exe（although this file is not intend to be used in StaticLib mode)
this .o has an dotnet main function defined, this main function will call InitializeRuntime to init dotnet runtime and then call __manged_Main (generated from C# Main function)

clang test-static.c \
  -o test-static \
  -force_load $net_static_lib_path \
  $native_path/libbootstrapper.o \  # add this to linker
  $native_path/libRuntime.WorkstationGC.a \
  $native_path/libSystem.Native.a \
  $native_path/libeventpipe-enabled.a \
  $native_path/libstandalonegc-enabled.a \
  $native_path/libz.a \
  ...
  ...

my dotnet static lib is build for Libaray mode, and does not has an C# Main function
so I change my C project's native main fucntion's name to __manged_Main to simulate an C# Main function

and it works
dotnet runtime is inited and my other C# UnmanagedCallersOnly function call does not crash any more

#include <stdio.h>

extern void TestStaticFunc();

extern int __managed__Main(int argc, char* argv[])
{
    printf("test static start...\n");

    TestStaticFunc();

    printf("test static successfully.\n");
    return 0;
}

but this workaround does not help my iOS App problem
because iOS App XCode project only use their own main function in ObjC
main function in libbootstrapper.o is ignored

libbootstrapper.o is from $HOME/.nuget/packages/microsoft.netcore.app.runtime.nativeaot.osx-arm64/.... and other static lib for dotnet runtime

[MichalStrehovsky]

libbootstrapper.o is used for executables. libbootstrapperdll.o is used in libraries. Try linking with that one.

clang test-static.c \
  -o test-static \
  -force_load net-static/bin/Release/$netVer/osx-arm64/publish/net-static.a \
+$native_path/libbootstrapperdll.o
  $native_path/libRuntime.WorkstationGC.a \
  $native_path/libSystem.Native.a \
  $native_path/libeventpipe-enabled.a \
  $native_path/libstandalonegc-enabled.a \
  $native_path/libz.a \
  $net10_extra \
  -lc++ \
  -framework Foundation -framework CoreFoundation

[Magicqy]

libbootstrapper.o is used for executables. libbootstrapperdll.o is used in libraries. Try linking with that one.

clang test-static.c
-o test-static
-force_load net-static/bin/Release/$netVer/osx-arm64/publish/net-static.a
+$native_path/libbootstrapperdll.o
$native_path/libRuntime.WorkstationGC.a
$native_path/libSystem.Native.a
$native_path/libeventpipe-enabled.a
$native_path/libstandalonegc-enabled.a
$native_path/libz.a
$net10_extra
-lc++
-framework Foundation -framework CoreFoundation

I test osx-arm64 native aot static lib link with libbootstrapperdll.o seems work corectly

---

but on iOS does not work

link with libbootstrapperdll.o iOS App crash stacktrace

* thread #1, queue = 'com.apple.main-thread', stop reason = signal SIGSEGV
    frame #0: 0x00000001085a06dc `TypeManager::GetClasslibFunction(ClasslibFunctionId)
    frame #1: 0x00000001094731fc `S_P_CoreLib_System_Runtime_EH__OnFirstChanceExceptionViaClassLib + 28
    frame #2: 0x0000000109473878 `S_P_CoreLib_System_Runtime_EH__DispatchEx + 152
    frame #3: 0x000000010947377c `RhThrowEx + 60
    frame #4: 0x00000001085f06c8 `RhpThrowEx + 188
    frame #5: 0x000000010947f758 `S_P_CoreLib_Internal_Runtime_CompilerHelpers_ThrowHelpers__ThrowIndexOutOfRangeException + 40
    frame #6: 0x000000010947f06c `S_P_CoreLib_Internal_Runtime_CompilerHelpers_StartupCodeHelpers__CreateTypeManagers + 492
    frame #7: 0x000000010947ecd0 `InitializeModules + 80
    frame #8: 0x000000010738e320 `InitializeRuntime() + 144
    frame #9: 0x00000001085a4148 `Thread::ReversePInvokeAttachOrTrapThread(ReversePInvokeFrame*) + 184
    frame #10: ....   call function from dotnet native aot static lib

with out libbootstrapperdll.o iOS App crash stacktrace like:

* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x0)
    frame #0: 0x00000001070d45b8 `profiling::Marker::GetCallbackRelaxed() const
    frame #1: 0x000000010820f6a0 `ThreadStore::AttachCurrentThread() + 84
    frame #2: 0x000000010820f4e0 `Thread::ReversePInvokeAttachOrTrapThread(ReversePInvokeFrame*) + 224
    frame #3 ....   call function from dotnet native aot static lib

iOS project linker files：

• libbootstrapperdll.o
• libSystem.Native.a
• libeventpipe-enabled.a
• libRuntime.WorkstationGC.a
• libstandalonegc-enabled.a

---

I tested on net10-preview, and crash stacktrace is

* thread #1, queue = 'com.apple.main-thread', stop reason = signal SIGSEGV
    frame #0: 0x000000010cac4d3c `RhpAssignRefArm64
    frame #1: 0x000000010dae2710 `InitializeModules + 208
    frame #2: 0x000000010b9f2320 `InitializeRuntime() + 144
    frame #3: 0x000000010ca77828 `Thread::ReversePInvokeAttachOrTrapThread(ReversePInvokeFrame*) + 184
    frame #4: ....   call function from dotnet native aot static lib

[Magicqy]

@MichalStrehovsky @agocke Today I found some new clues. I'm working on developing a native static library plugin for UnityEngine mobile game. This engine generates an iOS XCode Project, then built into an iOS App. The Project generated by the engine mainly consists of two parts: one is the target for the executable of the iOS App, which only contains the main function as the entry point of the application; the other is a target in the form of a .framework (dynamic library), containing all the game logic code, where my developed dotnet native static library plugin is placed.
I suspect the current issue lies in this usage scenario: a NativeAOT StaticLib is placed in an iOS .framework leading to a crash. When I tried calling the NativeAOT static library functions from the main function (outside .framework), it worked successfully. I will continue to troubleshoot this issue tomorrow. Your suggestions or solutions regarding this would be greatly appreciated.

[MichalStrehovsky]

Make sure you're passing all arguments to the linker that native AOT is passing when building a shared library. See https://github.com/dotnet/samples/tree/ada9a420b48174aa6ff3587dfe972268f7ca01ca/core/nativeaot/NativeLibrary#building-static-libraries for how (the "looking at the output generated by the LinkNative target" part). Building static libraries is unsupported because getting them to link properly is not trivial. I don't know what arguments are necessary on iOS (I don't have a mac to test with), but it sounds like the problem is that.

[agocke]

See https://learn.microsoft.com/en-us/dotnet/core/deploying/native-aot/ios-like-platforms/creating-and-consuming-custom-frameworks

It looks like you need to use install_name_tool and lipo to create a framework.