No coverage of how AllocHGlobal works under Linux. Documentation implies it is supported under Windows only.

[logical-intent]

Type of issue

Outdated article

Description

[Enter feedback here]
The documentation here says "This method exposes the Win32 LocalAlloc function from Kernel32.dll." I haven't tried using it under Linux but Copilot reckons it uses malloc there, so either it's hallucinating or this documentation is badly misleading and out of date.

Page URL

https://learn.microsoft.com/en-us/dotnet/api/system.runtime.interopservices.marshal.allochglobal?view=net-8.0

Content source URL

https://github.com/dotnet/dotnet-api-docs/blob/main/xml/System.Runtime.InteropServices/Marshal.xml

Document Version Independent Id

53418d00-f1c2-5cde-3843-b607b1dcf240

Article author

@dotnet-bot

[karakasa]

just fyi, under Unix-like systems, the method is implemented by NativeMemory.Alloc, essentially malloc.

https://github.com/dotnet/runtime/blob/128fdfd1fd982e277e969d5c7ca1710345f4f649/src/libraries/System.Private.CoreLib/src/System/Runtime/InteropServices/Marshal.Unix.cs#L59-L62

https://github.com/dotnet/runtime/blob/128fdfd1fd982e277e969d5c7ca1710345f4f649/src/libraries/System.Private.CoreLib/src/System/Runtime/InteropServices/NativeMemory.Unix.cs#L126-L137

[logical-intent]

just fyi, under Unix-like systems, the method is implemented by NativeMemory.Alloc, essentially malloc.

https://github.com/dotnet/runtime/blob/128fdfd1fd982e277e969d5c7ca1710345f4f649/src/libraries/System.Private.CoreLib/src/System/Runtime/InteropServices/Marshal.Unix.cs#L59-L62

https://github.com/dotnet/runtime/blob/128fdfd1fd982e277e969d5c7ca1710345f4f649/src/libraries/System.Private.CoreLib/src/System/Runtime/InteropServices/NativeMemory.Unix.cs#L126-L137

Thank you very much for your reply.

In some ways it might not hurt for the documentation to link to source code like this.

It is an odd default since memory allocated with C++ new[] is surely not safe to allow NativeMemory.Alloc to delete. I'm given to understand that even using delete rather than delete[] with memory allocated with new[] equates to UB in C++, so goodness knows how using free against that could be entirely safe... Even if you didn't care about destructors being called, you might be working with a different heap entirely.

The idea that you can confidently and safely deallocate memory on the managed side that was allocated on the native side seems like it must be deeply rooted in the idea that you're only P/Invoking Windows SDK functions. (Which I'm sure was the main original design goal behind P/Invoke, but is a confusing default otherwise.)

Your answer does at least give me confidence that I could thunk everything through a "pure C" layer if I wanted to do a load of extra copying of data just to satisfy the P/Invoke defaults. I found that a little bit hard to swallow and eventually (with significant and unnecessary difficulty - again I'd say the documentation leaving a lot to be desired) made enough sense of custom marshallers to write one that doesn't try to deallocate memory that I already have perfectly good lifetime management for on the native side.

It beggars belief that I had to do this though. I was sure there should be a simple attribute you could enlist to say whether or not you want to take ownership of and free the memory you're copying from when marshalling, but it seems there is not, and custom marshallers are the only way.

Anyway I do appreciate your reply and I hope this comment might help others (and the MS documentation team).

[bartonjs]

@AaronRobinsonMSFT @jkoritzinsky , is this you guys?

[AaronRobinsonMSFT]

@AaronRobinsonMSFT @jkoritzinsky , is this you guys?

Yes.

It is an odd default since memory allocated with C++ new[] is surely not safe to allow NativeMemory.Alloc to delete.

@logical-intent I'm not sure how this is "odd". There is no "default" allocator that gets assigned to new. The new keyword is the allocator and one must call the related delete, by default, everything else is undefined.

Your answer does at least give me confidence that I could thunk everything through a "pure C" layer if I wanted to do a load of extra copying of data just to satisfy the P/Invoke defaults.

Correct. The C ABI is stable whereas the C++ one isn't defined; therefore, interop with C++ is considered difficult in the best of circumstances and "unreliable" and non-portable outside of a C++ compiler being involved in the worst case.

It beggars belief that I had to do this though. I was sure there should be a simple attribute you could enlist to say whether or not you want to take ownership of and free the memory you're copying from when marshalling, but it seems there is not, and custom marshallers are the only way.

This has been asked, but relative to other requests it is rather rare. Using LibraryImport and creating a custom marshaller is the best approach here. We can help out clarify the best way to perform the interop call if you share the native signature.

In general the allocators on Marshal should be avoided unless one is running on Windows. The NativeMemory class is the recommended native memory allocator on non-Windows in all circumstances.

[logical-intent]

Hi @AaronRobinsonMSFT, thanks for your interest.

It is an odd default since memory allocated with C++ new[] is surely not safe to allow NativeMemory.Alloc to delete.

@logical-intent I'm not sure how this is "odd". There is no "default" allocator that gets assigned to new. The new keyword is the allocator and one must call the related delete, by default, everything else is undefined.

What I was trying to express was that it was odd that the interop APIs choose to take ownership of memory by default, apparently always for certain types (strings, arrays, maybe others? Is there a list somewhere?). Certainly it has some merit in that it's an opinionated approach and it seems to be pretty consistent. On the downside, it forces you to do a lot of data copying for no other reason than to avoid deallocating twice, and maybe even make a dedicated wrapper library to do so if you don't control the source code you're trying to interop with.

But fair enough, it's been this way for a long time and LibraryImport is a great improvement on DllImport in that now at least it allows us to step through generated code and understand that our program crashes because the generated code is trying to deallocate a static array (for example).

I just felt that it could be better signposted on the documentation side.

What we could really use in the documentation - if there's never going to be a nice simple attribute provided for this - is to find out what we ought to do once we do understand why it crashes.

Maybe the answer in that case is to provide a list of types with corresponding links to the source code of the "custom" (standard library) marshallers which you will have to copy all of, and then omit the Free() method from.

Or maybe the answer is to spell out in the documentation that you're always expected to do work on the native side to copy everything into heap memory using malloc for this list of types (e.g. arrays of native types, char* strings, etc.) because the policy is always to deallocate and - I guess - most likely you will not get much memory fragmentation from that since the allocations will be short-lived, and most likely the overhead of malloc, free, and memcpy are going to fall into the noise relative to things like converting your utf8 strings to utf16 for .NET. Is that the philosophy?

(Maybe it says that somewhere, and I just missed it...)

This has been asked, but relative to other requests it is rather rare. Using LibraryImport and creating a custom marshaller is the best approach here. We can help out clarify the best way to perform the interop call if you share the native signature.

I really appreciate the personalised attention, but at the same time I'd rather be taught how to fish than given a fish, if you know what I mean - but I found the learning curve presented by the documentation a bit steep on that front. I'm not much of a fan of Perl but I do love the motto of "make easy things easy and hard things possible" - it feels like on the P/Invoke front the emphasis is very much on the latter. This is what I came up with for dealing with a double[] that I didn't want deallocated:

    [CustomMarshaller(typeof(CustomMarshallerAttribute.GenericPlaceholder[]), MarshalMode.ManagedToUnmanagedOut, typeof(NonFreeingArrayMarshaller<,>))]
    [ContiguousCollectionMarshaller]
    public static unsafe class NonFreeingArrayMarshaller<T, TUnmanagedElement> where TUnmanagedElement : unmanaged
    {
        public static T[]? AllocateContainerForManagedElements(TUnmanagedElement* unmanaged, int numElements)
        {
            if (unmanaged is null)
                return null;

            return new T[numElements];
        }

        public static Span<T> GetManagedValuesDestination(T[] managed)
            => managed;

        public static ReadOnlySpan<TUnmanagedElement> GetUnmanagedValuesSource(TUnmanagedElement* unmanagedValue, int numElements)
            => new(unmanagedValue, numElements);
    }

It seems an awful lot of arcane incantations for a jobbing programmer with a reasonable competency in C++ and C# who just needs to tie them together -- especially given that the "implementation" this code provides is through what is omitted (i.e. an implementation of Free()). Because there is no attribute I can employ to direct the code generator to skip the Free, I have to reimplement the entire remainder of the marshaller.

But maybe as I said above, the real missing link in the documentation might just be to have a section about the generated code's approach to ownership of memory saying something like "Hey, it's not really much overhead to do some copying of data and in exchange for that you don't have to spend any more time becoming a C++/C# interop expert".

I'm not sure what the answer is - there is a lot of quite deep information in the interop documentation but I felt a bit let down by it on the "make the easy things easy" front. Does that make sense?

(Wow, I just realised this is something of a tangent from actual original issue I raised, sorry about that.)

[AaronRobinsonMSFT]

What I was trying to express was that it was odd that the interop APIs choose to take ownership of memory by default, apparently always for certain types (strings, arrays, maybe others? Is there a list somewhere?).

I think there is some confusion about expectations with respect to interop with memory managed languages. Converting native (unmanaged) memory to managed memory isn't possible. All managed memory must be allocated on the GC heap - so the GC can "manage" it. This means that all .NET Reference types must be allocated on the managed heap and during an interop operation, memory is often copied or pinned and then exposed to the unmanaged environment. On the way back, native to managed, the returned memory is, optionally, converted into managed reference types and thus copied into the managed heap and then the native side is deallocated. .NET has value types, structs in C#, that can be made blittable. Note that one can always do interop in a non-copiable way by falling back to void* or for blittable cases, something like double* to represent an array. This can be abstracted over by using the Span<T> type.

It seems an awful lot of arcane incantations for a jobbing programmer with a reasonable competency in C++ and C# who just needs to tie them together
I'm not sure what the answer is - there is a lot of quite deep information in the interop documentation but I felt a bit let down by it on the "make the easy things easy" front. Does that make sense?

For .NET interop guidance, we have a best practices guide, but interop in general is a very complicated and non-trivial problem. Almost every language is different - C++, Swift, Objective-C, Ada, Rust, Java, etc. All have nuances that we try to abstract over and the only one that is generally "simple" is C, but there are rough edges. All the others come with many gotchas and there is rarely any authoritatize document on all of it, because interop means leaving the nicely designed environment for another and that is, by design, not generally a pri0 scenario for a language/platform.

It sounds like you would like to expose an native allocated double[] from C/C++. Let's imagine the C signature is as follows:

void M(size_t len, double* arr);

In C#, the SpanMarshaller<T,TUnmanagedElement> could be used to wrap the arr and convert that to Span<double>.