Brief

A small repository to test out some scenarios related to writing a custom dotnet host in c++. There is an official article about how to make a dotnet host using the nethost api, that comes with a sample repository in github.

However, the sample project did not contain an example of the scenario that I was interested in, and I had some trouble making it work with Rider.

There is ample documentation on how to call native code from C#, however I have found little information about calling functions in the host in a native hosting scenario. If I host a dotnet runtime in my program, that is probably because I wish to script parts of that program, which makes it natural to assume that I also want to call functions and send data back and forth. The current sample project and documentation did not explain how to call C++ functions in the host program from the hosted dotnet.

This repository is an attempt to make the example work with Rider and add more examples for my own reference. I have also taken the liberty to attempt to address some of the warnings that clang-tidy gives.

Disclaimer

The purpose of this repository is to provide some hands-on examples and gather some links to relevant documentation. It is not a demonstration of best practices, nor a demonstration of how to create pretty code. Feel free to use this code however you wish, but let's be responsible with random code that we find on the internet :)

Documentation

I haven't found any real documentation on this topic, apart from the article on MS Learn and the sample repository. There is also this design document on the native hosting feature that seems to document the functions used in the sample.

To invoke functions provided by the host there seem to be basically two ways to go (that I'm aware of):

1. Provide function pointers to dotnet that expose the desired functionality. This was outlined in the discussion here.
2. Declare functions with internal linkage (__Internal) and do a P/Invoke into the host binary. See the discussions in this and then this github issue.

How to Run

All compilation and running is through the project called NativeHost that shows up as a c# project in Rider.

Provide Function Pointers to Hosted Dotnet Code

Native Host Exposing Functions to Managed Assembly

I got this idead from a reply on this github issue, that suggested that you can do this by passing in a function pointer, without using the Marshal.GetDelegateForFunctionPointer that shows up if you google the topic.

In short, this can be achieved by declaring a method like so in c#:

[UnmanagedCallersOnly]
public static unsafe void TestFnPtrWithArgs(delegate* unmanaged<int, double> fn_from_cpp) { ... }

This will provide the c# code with a delegate that it can use normally like this:

double @return = fn_from_cpp(20);

Which would then invoke some code that is defined in the host. The c# code can also store this delegate for later use.

To expose a function from the c++ side we would need to first define the callback type:

typedef void (CORECLR_DELEGATE_CALLTYPE *send_callback_to_dotnet_fn)(double_t(*fn)(int32_t));
send_callback_to_dotnet_fn callback;

We can then get a pointer to the function in c#:

typedef void (CORECLR_DELEGATE_CALLTYPE *send_callback_to_dotnet_fn)(double_t(*fn)(int32_t));
send_callback_to_dotnet_fn callback;

rc = load_assembly_and_get_function_pointer(
    dotnetlib_path.c_str(),
    dotnet_type,
    STR("TestFnPtrWithArgs"),
    UNMANAGEDCALLERSONLY_METHOD,
    nullptr,
    (void**)&callback);

Now we can either give the c# code a pointer to a free function:

callback(&test_fn_arumgents_and_returns);

or we can give it a lambda function:

auto fn = [](int32_t i) -> double_t
{
    std::cout << "[C++] A lambda recieved " << i << " from dotnet!" << std::endl;
    return static_cast<double_t>(i) + 3.14;
}; 
callback(fn);

That's it, we now know how to expose functions for consumption by the hosted dotnet runtime!

Marshalling Strings

There ample is documentation on how to handle string types when doing a P/Invoke, however I couldn't find anything that mentions the case of a native host calling functions directly or exposing functionality via function pointers like we are doing here. Luckily it seems the same principles apply in our case as well!

The signature of a method in c# to send and receive a string would be the following:

delegate* unmanaged<IntPtr, IntPtr> str_fn

And the entire method in the test application looks like this:

[UnmanagedCallersOnly]
public static unsafe void TestStringInputOutput(delegate* unmanaged<IntPtr, IntPtr> str_fn)
{
    Console.WriteLine($"[C#] Entering {nameof(TestStringInputOutput)}");

    string str_from_cs = "String from c#";
    IntPtr cpp_str = str_fn(RuntimeInformation.IsOSPlatform(OSPlatform.Windows)
                                ? Marshal.StringToCoTaskMemUni(str_from_cs)
                                : Marshal.StringToCoTaskMemUTF8(str_from_cs));
    
    string cs_str = RuntimeInformation.IsOSPlatform(OSPlatform.Windows)
        ? Marshal.PtrToStringUni(cpp_str)
        : Marshal.PtrToStringUTF8(cpp_str);
    
    Console.WriteLine($"[C#] String from c++: {cs_str}");
}

First we prepare a string for sending to the native code by calling either Marshal.StringToCoTaskMemUni or Marshal.StringToCoTaskMemUTF8 depending on whether we are on Windows or not. We then send this string to the native host using the provided callback.

In the next step we receive a string from the native host, which we convert to something that dotnet can understand, again depending on which operating system we are on. Finally we write this string to output. In the console we should see something like this:

[C#] Entering TestStringInputOutput                           
[C++] C# sent the following string: String from c#            
[C#] String from c++: This string is from c++ :)  

We may need to think about the character set that we are using, which is described in more detail here. To this end I added the following attribute to the c# code:

[module: System.Runtime.InteropServices.DefaultCharSet( CharSet.Unicode )]

If the strings show up as empty when printing, we can play around with this and the types that we use on the c++ side as well.

An interesting thing that I noticed was that I could send (but not receive) "raw" strings to c++ using the following signature:

delegate* unmanaged<string, IntPtr> str_fn

To properly receive this in c++ I had to accept it as a char const* instead of a wchar_t const*:

callback( [](char const* str) -> wchar_t const*
{
    std::cout << "[C++] C# sent the following string: " << str << std::endl;
    return STR("This string is from c++ :)");
});

Not sure to what extent this would work however or what the best practices are here. Do we always use the Marshal class to prepare our strings for interop, or are there some scenarios where it is OK to pass "raw" strings like this?

Functions for String Marshalling

There are many methods available in the Marshal class, but in this example we have used the following ones.

c# <= c++	c# => c++
PtrToStringUni	StringToCoTaskMemUni, StringToHGlobalUni
PtrToStringUTF8	StringToCoTaskMemUTF8

Not quite sure what the difference between StringToHGlobalUni and StringToCoTaskMemUni is - both are listed as the inverse of PtrToStringUni in the documentation

P/Invoke With Internal Linkage

I'm not sure if "internal linkage" is the correct technical term here, but hopefully the reader understands what I mean. It refers to registering a function that can be found in the binary/dll of the host, and then making a P/Invoke.

Funnily enough, I got this idea from studying the Mono documentation about embedding, which I find is not too bad, actually. This lead to the question of how this can be done in dotnet core, and this github issue. It turns out that we can now do the same thing in dotnet core.

P/Invoke - Quick Start Guide

To begin we first create a function with the desired functionality in C++:

extern "C" __declspec(dllexport) void print_simple_message() {...}

In C# we can then declare a method with the same return value and parameters. We then decorate it with the DllImport attribute and specify __Internal as the library name:

[DllImport("__Internal", EntryPoint = "print_simple_message")]
private static extern void PrintSimpleMessage();

This will require us to write some additional code as well for marshalling parameters etc. However, from dotnet 7.0 we can take advantage of source generators to generate the marshalling code. Doing this will change the method definition slightly:

[LibraryImport("__Internal", EntryPoint = "print_simple_message")]
private static partial void PrintSimpleMessage();

We now use the attribute LibraryImport and we also need to make the method partial so that the source generators can do their thing. When testing I also ran into the error SYSLIB1051 which was solved by adding the following attribute to the assembly:

[assembly: System.Runtime.CompilerServices.DisableRuntimeMarshallingAttribute]

More on P/Invoke

The example code in nativehost.cpp contains a few examples of functions that receive data from dotnet and perform some logic (represented by print statements in the example). The reader should hopefully be able to figure out how to adapt them to his/her own project.

The examples show how to deal with simple parameters, which is easy to set up thanks to the source generators - we almost don't need to do any thinking at all. I have also included an example of a struct parameter:

struct ComplicatedParamStruct
{
    int SomeOption;
    double ValueOfOption;
    bool DoComplicatedThingy;
};

This can be received from dotnet either as a pointer, reference or by copying:

extern "C" __declspec(dllexport) void print_struct_pointer(ComplicatedParamStruct* params);
extern "C" __declspec(dllexport) void print_struct_reference(ComplicatedParamStruct& params);
extern "C" __declspec(dllexport) void print_struct_copy(ComplicatedParamStruct params);

To make this work in C# we need the corresponding struct definition:

[StructLayout(LayoutKind.Sequential)]
struct ComplicatedParamStruct
{
    public int SomeOption { get; set; }
    public double ValueOfOption { get; set; }
    public bool DoComplicatedThingy { get; set; }
};

as well as the method definitions:

[LibraryImport("__Internal", EntryPoint = "print_struct_pointer")]
private static partial void PrintStructPointer(ref ComplicatedParamStruct @params);

[LibraryImport("__Internal", EntryPoint = "print_struct_copy")]
private static partial void PrintStructCopy(ComplicatedParamStruct @params);

[LibraryImport("__Internal", EntryPoint = "print_struct_reference")]
private static partial void PrintStructReference(ref ComplicatedParamStruct @params);

Note that the parameter is declared with the ref keyword for both the pointer and the reference cases.

P/Invoking with Strings

String may require some more consideration, but here we can also utilize the source generators to make our life easier. We start with a simple logging function:

extern "C" __declspec(dllexport) void native_log(char const* message) {...}

Note that the function takes a char and not a char_t (which in this project is a char on linux and wchar on windows). The corresponding method declaration in C# is:

[LibraryImport("__Internal", EntryPoint = "native_log", StringMarshalling = StringMarshalling.Utf8)]
private static partial void NativeLog(string message);

What is different from before is the property StringMarshalling = StringMarshalling.Utf8, which generates code to marshal the string as a Utf-8 string. We can also marshal as Utf-16 or Custom.

We also have the possibility of using the attribute [MarshalAs(...)] on parameters and return type, as described here.

In addition, we may define our own classes to handle the marshalling if we need to. To do this we declare a class with the following attribute:

[CustomMarshaller(typeof(string), MarshalMode.Default, typeof(CustomMarshallerWithLibraryImport))]

We can then define a method for marshalling in the way we need to:

public static UInt32* ConvertToUnmanaged(string? str)
{
    if (string.IsNullOrEmpty(str))
    {
        throw new InvalidOperationException("Unable to marshal object");
    }

    return (UInt32*)(RuntimeInformation.IsOSPlatform(OSPlatform.Windows)
        ? Marshal.StringToCoTaskMemUni(str).ToPointer()
        : Marshal.StringToCoTaskMemUTF8(str).ToPointer());
}

So depending on the OS we marshal either as Utf-8 or Utf-16. There are many ways to define a custom marshaller, please refer to the official documentation and this design document for further reading.

Limitations

This has only been tested on Windows and Rider so far. In the future I hope to get the time to see if it runs on Linux as well. I would also like to see if I can remove all the "solution"-related things and convert it to using CMake.

Todo

• Add more of my own examples
• See if it is possible to convert to a CMake project
• Make sure this compiles and runs on Linux

License and acknowledgements

This project is licensed under the MIT license. It contains modified code from the official dotnet samples repository found here: https://github.com/dotnet/samples/tree/main/core/hosting

Further Reading

• Native Hosting
• P/Invoke Source Generation
• Custom Marshalling with Source Generation
• User Type Marshalling - v2
• https://github.com/KevinGliewe/embedded_dotnet_runtime_examples
• https://github.com/AaronRobinsonMSFT/DNNE/tree/master