System.Runtime.InteropServices.MarshalDirectiveException: Cannot marshal 'parameter #1': Non-blittable generic types cannot be marshaled

[ethindp]

@Nihlus I have a C API that has a lot of void* pointers. It also uses callbacks in certain places. Pretty much all of the void* parameters are nullable. But the API also accepts arbitrary float* and other typed arrays, with a separate length parameter. However, I want to use Span<T> and Memory<T> instead of raw arrays if at all possible. However, whenever I try to activate the interface, I get this error:

  Failed TestDoubleRefcount [112 ms]
  Error Message:
   System.Runtime.InteropServices.MarshalDirectiveException : Cannot marshal 'parameter #1': Non-blittable generic types cannot be marshaled.
  Stack Trace:
     at AdvancedDLSupport.NativeLibraryBuilder.ActivateClass(String libraryPath, Type baseClassType, Type[] interfaceTypes)
   at AdvancedDLSupport.NativeLibraryBuilder.ActivateClass[TClass,TInterface](String libraryPath)
   at AdvancedDLSupport.NativeLibraryBuilder.ActivateInterface[TInterface](String libraryPath)
   at Synthizer.Tests.Utils.FFIActivator.ActivateFFIInterface() in /home/ethin/source/SynthizerSharp/SynthizerSharp.Tests/Utils.cs:line 35
   at SynthizerSharp.Tests.UnitTest1.TestDoubleRefcount() in /home/ethin/source/SynthizerSharp/SynthizerSharp.Tests/UnitTest1.cs:line 38

I'm confused because according to many resources that I've found, including the MS docs themselves, they imply that Span<T> and Memory<T> should be able to be passed to unmanaged code or across the FFI boundary. However, this doesn't appear to be the case. (In many places, I'm doing things like Memory<byte>? and Span<byte>? to indicate that these can be null.) Am I misusing these types? And, if so, how do I use them properly? Should I just revert to normal arrays?

[ethindp]

@Nihlus I've resolved this problem by just switching back to normal T[] arrays, but now my app is crashing (as in, the host process is crashing with a bizarre stack trace). My unit tests are also breaking because the host test harness is also breaking. Trying to debug this is not going well. I'm unsure how I should go about resolving this problem since I'm on Linux, and LLDB isn't much help. :-( Any advice would be appreciated. I'll keep digging around and trying different things in the meantime.

[Nihlus]

This may be due to how ADL marshals (or rather, doesn't marshal) spans under the hood - spans weren't widely adopted when I last fiddled with the code, and it's probably a feature gap right now. I'll see what I can do, though.

[Nihlus]

One option is to declare the parameter as a ref T and use MemoryMarshal.GetReference() or T* and pin the span.

[ethindp]

@Nihlus Definitely isn't something I considered (or even knew about). The crashing problem appears to be due to something modifying memory that it shouldn't -- on Windows I get an access violation exception and on Linux I just get a crash. It oddly appears to be a function that (I don't think) modifies its parameter. I might need to ask the library author about that though. Changing that to a ref T doesn't seem to solve the problem so...
As for the Span<T>/Memory<T> issue, thanks for the info -- will try that.

[Nihlus]

@ethindp I've gone through the code, and spans are actually marshalled - they get converted into a pinned reference and then into a pointer. If they don't work for you, there's probably something else at play. Can you share the code at issue?

[ethindp]

@Nihlus Sure! Here are the files in question:

• C library headers (only the top-level files (synthizer.h and synthizer_constants.h) in here are relevant)
• C# FFI bindings interface
• Current unit tests

This doesn't properly compile right now; for some reason, Roslynn dislikes this line (within the CustomStreamDefinition struct, l. 252-260 of Synthizer.FFI.cs):

    public Span<byte>? userdata;

Oddly, it likes it if I change it to:

    public Memory<byte>? userdata;

Also, if you could, I would appreciate it if you could check my unions (those annotated with StructLayout). I've attempted to do it myself, both in my head as an attempt to calculate the actual size, and by using the sizeof operator. The union I'm particularly concerned about is AutomationCommandParameters (its C partner is syz_AutomationCommandParams). Right now I have it declared like this:

[StructLayout(LayoutKind.Explicit, Pack = 1)]
public struct AutomationCommandParameters
{
    [FieldOffset(0)]
    public AutomationAppendPropertyCommand AppendToProperty;
    [FieldOffset(sizeof(AutomationAppendPropertyCommand))]
    public AutomationClearPropertyCommand ClearProperty;
    [FieldOffset(sizeof(AutomationAppendPropertyCommand) + sizeof(AutomationClearPropertyCommand))]
    public AutomationSendUserEventCommand SendUserEvent;
}

But I don't know if this will actually work or if I need to subtract 1 from the sizes, or if it has to be a constant. I'm new to FFI in C# so...

[Nihlus]

Looking at your code, there are several mismatches between the C# side and the native side.

Overall tips:

• May want to check the size of int on your platform - it's not like C# where it's guaranteed to be 32 bits, C/C++'s keywords only specify a minimum size
• Check all your usages of Span - it's fine as a parameter when you have a pointer of the same type, but cannot be used as a drop-in replacement for a pointer field in a struct.
• Span is also a stack-only type - unless you declare a struct as ref struct, you cannot have a Span field in it.
• May want to check your string parameters and annotate them with the marshalling to use. It tends to differ between platforms and libraries.

syz_Event (AutomationEvent)

• Source and Context should be declared as ulong (unsigned long long has a size of at least 64 bits)

syz_AutomationCommandParams (AutomationCommandParameters)

• Union is declared incorrectly - what you have is no different than not using FieldOffset at all. Set FieldOffset(0) on all members in order to make them overlap (the default in C/C++).

syz_AutomationCommand (automationCommandData)

• Same here with Target - wrong size

syz_AutomationPoint (AutomationPoint)

• Values cannot be declared as a span - an array as a field in C/C++ is stored inline, while a Span is inherently a pointer. Declare it as fixed double Values[6] or as 6 individual fields.

syz_SineBankConfig (SineBankConfig)

• Same here, Waves cannot be a span since there's no association with the length field that comes after it. Make it a SineBankWave*.

UserDataFreeCallback

• Usage of Span<byte> here is slightly suspicious - void* means literally any kind of memory, not just a sequence of bytes.

StreamDestroyCallback

• See above

syz_CustomStreamDef (CustomStreamDefinition)

• Cannot have span field userdata - same as before, spans are a pointer/length pair. Declare it as void*.

[ethindp]

@Nihlus Thank you! I've been working on changing the types to conform closer to the C spec, and thus far I've made some good progress and taken into account your suggestions. As an example, for the return types where C's int type is used, I've just switched it to use nint. And for unsigned int, I'll change that to be nuint. I am unsure about a couple things. There are a few places in the header that have pointer arrays of non-void pointer types (mainly, float*, char* (though that's just bytes and not actually a string -- the code for loading files from encoded data function), etc. Should I change that to use a raw float*/sbyte* pointer, or would Memory<Byte>/Memory<Float> martial correctly in this case?

[Nihlus]

int tends to be 32-bit, even on 64-bit platforms - it's mostly pointer size you need to worry about.

I would use raw pointers for struct members, and then have a higher-level managed layer that wraps it in a nicer type for C# consumption.

[ethindp]

@Nihlus I mean, I could use UIntPtr for the void* pointers, and for the pointers in general, but why for all the others? Is it because Marshalling could change how they're represented at any time?