Native AOT CLI Experience Improvements in .NET 8

[agocke]

The Native AOT CLI (SDK) experience has a lot of rough edges and room for improvement.

• Improve experience when using PublishAot with unsupported RID #73678
• NativeAOT: dwarfdump --verify produces thousands of warnings and errors #72552
• Improve static library publishing #70277

[dotnet-issue-labeler]

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

[ghost]

Tagging subscribers to this area: @agocke, @MichalStrehovsky, @jkotas
See info in area-owners.md if you want to be subscribed.

Issue Details

---

The Native AOT CLI (SDK) experience has a lot of rough edges and room for improvement.

• Improve experience when using PublishAot with unsupported RID #73678
• NativeAOT: dwarfdump --verify produces thousands of warnings and errors #72552
• Improve static library publishing #70277

Author:	agocke
Assignees:	-
Labels:	area-NativeAOT-coreclr
Milestone:	-

[Xylobol]

Sorry to bump this with a few people on it, but I'm interested in helping with this. I have a few projects I want to work on that depend on building native libraries from C# code, but the experience has been frustrating. I'm not very experienced with "compiler stuff", nor have I landed changes in .NET, but I'd love to learn way more about both, and this seems like a great opportunity to do so. If you've got time, please send some pointers for where work can be done on these.

[agocke]

@Xylobol The issues linked in there might be a good place to get started, but I'd love to make your experience better.

Maybe you could try writing down what you wanted to and what you expected the experience to be, then the problems with it? Writing native libraries is a pretty new thing for the dotnet toolchain so I expect there are some rough edges, but don't have a good sense of what people will run into.

[Xylobol]

I'm trying to get a "native" shared library that I can dynamically load from a C/C++ program running on Linux. The generation itself seems to be fine on the .NET side - I can inspect the compiled library and see my methods. I haven't been able to consume the library though - per #70277, there's a ton of extra things to link when building. A complete example using CMake would be very helpful.

[Xylobol]

I got it working and prepared an example solution over at https://github.com/Xylobol/DotNet-NativeAOT-Library-Example. I'm likely going to add extra information on writing headers and calling from C/C++ later, maybe in addition to how to deal with static libraries.

EDIT 12/24/22, a few minutes after posting: A source generator for header files would be killer

[Xylobol]

A few more questions:

• How do I handle a callback passed in by native code? e.g. I have an exported C# method void DoSomething(SomeType callback) and native code wants to give it a callback
• Any reason I can't use byte[]?

Merry Christmas or whatever you all may celebrate, and happy New Year!

[Xylobol]

In the new Christmas tradition of answering my own questions: for any other explorers, use delegates for the first part of that last post!

In the exporting lib:

public static partial class Class1
{
    // Callback function which is called from the unmanaged code
    [UnmanagedCallersOnly]
    private static void TestCallback()
    {
        Console.WriteLine("Hello from the test callback");
    }
    
    [UnmanagedCallersOnly(EntryPoint = "GetFunctionPointer", CallConvs = new[] { typeof(CallConvCdecl) })]
    private static unsafe delegate* unmanaged<void> GetFunctionPointer()
    {
        return &TestCallback;
    }
}

In the program you're calling from, in this case also written in C#:

public static partial class Class1
{
    private delegate void TestDelegate();
    
    [LibraryImport("whatever.dll", EntryPoint = "GetFunctionPointer")]
    [UnmanagedCallConv(CallConvs = new Type[] { typeof(System.Runtime.CompilerServices.CallConvCdecl) })]
    private static partial nint GetFunctionPointer();

    private static readonly TestDelegate Delegate = Marshal.GetDelegateForFunctionPointer<TestDelegate>(GetFunctionPointer());

    public static void RunTheFunction() => Delegate();
}

---

A lot of this seems like documentation that assumes you already know what you're doing - I'd love to help with that, whether that's writing or just giving input to someone else who is.

[jkotas]

    private static nint GetFunctionPointer()
    {
        var testDelegate = new TestDelegate(TestCallback);
        return Marshal.GetFunctionPointerForDelegate(testDelegate);
    }

This method has "callback on collected delegate" bug that will cause it to crash intermittently. Marshal.GetFunctionPointerForDelegate documentation
says "You must manually keep the delegate from being collected by the garbage collector from managed code. The garbage collector does not track references to unmanaged code.". It is possible to fix the bug by keeping the delegate alive as the documentation suggests, however it is better to use function pointers here instead. The library code rewritten using function pointers would look like this:

public static partial class Class1
{
    private delegate void TestDelegate();
    
    //Callback function which is called from the unmanaged code
    [UnmanagedCallersOnly]
    private static void TestCallback()
    {
        Console.WriteLine("Hello from the test callback");
    }
    
    [UnmanagedCallersOnly(EntryPoint = "GetFunctionPointer", CallConvs = new[] { typeof(CallConvCdecl) })]
    public static unsafe delegate* unmanaged<void> GetFunctionPointerNative() => &TestCallback;
}

Similarly, you can use function pointers for the calling code as well.

[Xylobol]

Wow, that's really clean - thanks for showing me! I'll update my example soon.

[MichalStrehovsky]

I pushed static library publishing to .NET 9. We don't currently have plans to encourage people to do static libraries. It will be more realistic if we do #83611 - that one avoids the issue of "you need to match the toolset that was used to compile the unmanaged parts of the runtime or bad things might happen".

There's nothing else left here, so closing.