Support for new LibraryImport attribute

[kkukshtel]

Hello! Finally got this working and really love it. However I'd love to be able to use the new LibraryImport attribute instead of DLLImport (as well as its new parameters). I think exposing this as a configuration option if using preview/latest would make the most sense. Maybe being able to specify default params as well?

[tannergooding]

Today ClangSharp generates fully blittable bindings and so there is no benefit or need to using LibraryImport, it just causes a source generator to run which does "nothing" and generates an identical signature to what ClangSharp has already generated.

Supporting non-blittable bindings gets into the complexity I covered in #428 (comment) and while it isn't impossible for such a thing to be added, it represents a great deal of complexity to the codebase and has been considered "out of scope" up until this point.

[kkukshtel]

Okay I see. I thought I had read in the release notes that it was faster, but you may be right that it only matters if the code uses marshalling.

[tannergooding]

Right, when marshalling is involved it is faster than using the runtime's "built-in" marshalling. But for blittable bindings its effectively a no-op and so just spends extra compile time cycles unnecessarily.

[jwosty]

I'm using ClangSharp for some bindings, and wrapping them in slightly higher abstractions, in particular using SafeHandle-derived types, and then wrapping that in one more layer with actual objects. Manually dealing with the marshalling code in that middle layer is pretty tedious. Is there a better way?

One idea I've considered trying is building another codegen layer over the output of ClangSharp.PInvokeGenerator (just seems like a big undertaking)

Maybe another way would be to have something similar to LibraryImport, but which can be pointed at an existing method instead of generating its own. That would be a suggestion for a different repo of course

[kkukshtel]

Just based on my experience with this library I would say that if are doing intermediary marshaling on top of these bindings, you're better off just using LibraryImport on top of function bindings that are meant to be marshaled. Writing your own marshaling layer means you're effectively competing with the C# team on marshaling perf, and if it were me I'd assume the C# team can write better managed marshaling code than I could.

[tannergooding]

Is there a better way?

Not really. C/C++ isn't strongly typed in many cases (it is very typedef heavy) and doesn't have a built in way to express the additional semantics that get used in practice for many APIs.

For example, consider what is simply char* in C/C++. Given that, is the correct C# signature:

• string, marshalled as ASCII
• string, marshalled as ANSI
• string, marshalled as UTF8
• byte*
• out byte
• ref byte
• in byte
• ref readonly byte
• out byte
• byte[]
• one of the above, but using sbyte
• etc

The answer to that question is that its all contextual. Some libraries use typedefs which have semantic meaning per library (like LPSTR or LPCSTR in Win32). Some can differ based on a neighboring parameter (such as having a int32_t count parameter elsewhere in the method that is doc'd to correspond to that char*). Some of the semantics differ based on target platform/ABI. For example, many platforms treat char as signed char by default and therefore sbyte is more correct mapping in C#; but some treat it as unsigned char instead. For some ABIs the difference between signed vs unsigned doesn't matter, but for others (like RISC-V) it does. It can also impact other code when implicit casts, comparisons, or other operations get involved.

So, without some kind of additional metadata, such as reading the documentation or the library using a higher level feature such as SAL annotations, there is no way for ClangSharp to determine what is "correct". Additionally, even when SAL annotations do exist, there are often subtleties that still can't be codified. For example, while annotations exist on many methods, they are often missing from structs. Likewise there are cases like lpszClassName in WNDCLASS where it can be a null terminated string up to 256 characters in length -or- it can be an ATOM returned by RegisterClass.

If you're creating a higher abstraction, you're going to ultimately have your work cut out for you in terms of handling marshalling and defining "correct" bindings based on the available documentation, specifications, or annotations provided for that unique library. Some popular ones libraries like Win32 or Vulkan or ... have projects that aim to provide these higher level abstractions and make it "friendlier" to typical C# users (CsWin32, Silk.NET, TerraFX.Interop.*, Vortice and so on are some examples). Each brings its own unique flavor of those bindings and what they think is "best" for doing bindings

---

My own view is that having the higher level abstractions and forcing things like SafeHandle is undesirable. It forces users into what is typically allocation heavy and costly abstractions that they can't avoid. I believe it's better to expose the raw bindings and then incrementally build better support on top more piecemeal as needed

You can see this in ClangSharp with its 3-tiered approach.

1. You have ClangSharp.Interop which provides the very raw 1-to-1 bindings with direct access to the blittable P/Invokes: https://github.com/dotnet/ClangSharp/tree/main/sources/ClangSharp.Interop/clang
2. You then have some basic APIs that makes it easier to locate methods in an "object oriented" way (P/Invokes that operate on cursors are exposed on CXCursor and so on) and which do the conversion, wrapping, or basic marshalling of things like pointers to strings/arrays/spans, or vice versa: https://github.com/dotnet/ClangSharp/tree/main/sources/ClangSharp.Interop/Extensions
3. You finally have the proper managed abstraction. It exposes an API surface that more closely mirrors what you would find in C++. It uses .NET like names, manages lifetimes via IDisposable, wraps handles, gives you classes and convenience methods, etc: https://github.com/dotnet/ClangSharp/tree/main/sources/ClangSharp

With this approach, users have the option of what works best for them and you aren't forcing additional abstractions or indirections between the layers. Things end up exposed in a way that allows it to be handled quite optimally so that rather than having SomeClass which contains 3-4 SafeHandle which each abstract some native handle/allocation, you just get SomeClass which directly abstracts 3-4 native handles/allocations and which itself acts as the higher level "safe handle" (just without actually being a safe handle).

[jwosty]

Interesting. I like the idea of the multi-tier approach, and that's why I'm trying to accomplish -- reminds me of the onion architectural pattern from F#.

For my specific case, I'm tinkering with Apple API bindings from scratch, starting with bits of CoreFoundation, as an experiment, to see how one might improve upon some of Xamarin's shortcomings. (I understand that this is a gigantic can of worms -- hence the experimental nature, probably won't be used for anything serious)

I'm not asking for ClangSharp to try do anything with strings for me, as I understand their messiness. I'm fine with manually bridging between .NET strings and CFString, since thinking about it carefully case-by-case is of course necessary here.

The main thing I'm trying to figure out is if there's a way to automate away at least some of the tediousness of working with all of the Apple "object" lifetimes (just pointers, like CFStringRef, whose lifetimes are managed with reference counting via CFRetain and CFRelease). There are only a handful of patterns to choose from for each method argument/return, and they are typically very well-defined (for example the Create pattern, Copy pattern, etc).

My first attempt was actually to just do it all myself with LibraryImport, to get the automatic custom marshalling, at the cost of having to manually bind everything. Right now I'm trying out ClangSharp, to get automatic binding generation, but realizing the cost is marshalling boilerplate. I just wish there were a way to get the best of both worlds!

[Micke3rd]

thank you for asking & answering this topic.
I would like to leave the link to the related runtime discussion dotnet/runtime#60595 in order to shorten the search up to this point