Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions IgnoredWords.dic
Original file line number Diff line number Diff line change
Expand Up @@ -28,11 +28,13 @@ blockdiag
blog
bool
borked
buf
buildbinoutput
buildtransitive
builtinop
byref
byval
callee
canonicalization
castable
cibuild
Expand All @@ -56,6 +58,7 @@ dllimport
docfx
docfxconsole
dotnet
downcasts
endian
endianess
enum
Expand Down Expand Up @@ -97,10 +100,12 @@ malloc
marshallers
marshalling
materializer
materializers
memcopy
memcpy
memset
metadata
minimalistic
Mips
msbuild
msg
Expand All @@ -111,6 +116,7 @@ nounwind
nuint
nullability
Nullable
numerics
Nvidia
online
optimizenone
Expand Down Expand Up @@ -147,6 +153,7 @@ Subrange
Sym
telliam
templated
templating
tl
trx
typdef
Expand All @@ -162,6 +169,7 @@ undiscoverable
Unhandled
uniqued
uniqueing
unmanaged
unmarshal
unreferenced
Unshipped
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -47,12 +47,7 @@ public void LibLLVMGetKnownAttributeNamesTest( )
var actualNames = new LazyEncodedString[len];
for(int i = 0; i < len; ++i)
{
// https://github.com/microsoft/testfx/issues/5543
#pragma warning disable MSTEST0037 // Use proper 'Assert' methods
Assert.IsTrue( ppData[ i ] is not null );
#pragma warning restore MSTEST0037 // Use proper 'Assert' methods

var les = LazyEncodedString.FromUnmanaged(ppData[i]);

actualNames[ i ] = LazyEncodedString.FromUnmanaged( ppData[ i ] )!;
}
Expand Down
23 changes: 13 additions & 10 deletions src/Interop/LlvmBindingsGenerator/ReadMe.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ IN C# already as part of P/Invoke generation so there wasn't much point in conti
that. (Though there is something to be said for use as a starting point...)

## Split implementation
This app was subsequently split into to implementations that live in distinct repositories
This app was subsequently split into to implementations that now exist in distinct repositories
1) Generates the EXPORTS.g.def for the Windows DLL generation from the LLVM + LIBLLVM headers
1) This version lives in the [LibllVM repository](https://github.com/UbiquityDotNET/Llvm.Libs)
2) Generates the "safe handle" C# code from the LLVM + LIBLLVM headers
Expand All @@ -18,17 +18,18 @@ This app was subsequently split into to implementations that live in distinct re
### Common implementation
While there is a common implementation between the implementations (They started as simply the
same code and commenting out the functionality not desired) they have and will diverge over
time though anything in the core parsing of headers and general code generation from templates
time, though anything in the core parsing of headers and general code generation from templates
is likely to remain. (It may be viable to support a common library for this scenario but this
is ONLY necessary when the native side of the interop library changes)

## Usage
> [!IMPORTANT]
> This project has a dependency on the `CppSharp` library which ONLY supports the `X64`
> architecture but the generated wrappers are NOT dependent on a particular architecture.
> This limits the environments that can be used to generate the sources. To simplify that
> This limits the environments that can be used to generate the sources. To simplify that,
> the generated sources are placed into source control but generated off-line by a developer.
> A developer machine doing this ***MUST*** be X64 or the tool can't run.
> A developer machine doing this ***MUST*** be X64 or this tool can't run. This is a limitation
> defined by a dependent library.

`LlvmBindingsGenerator -l <llvmRoot> -e <extensionsRoot> -h <HandleOutputPath> [-Diagnostics <Diagnostic>]`

Expand Down Expand Up @@ -58,13 +59,13 @@ not used.

#### Roslyn Source Generators - 'There be dragons there!'
Roslyn allows source generators directly in the compiler making for a feature similar to C++
template code generation AT compile time. However, there's a couple of BIG issue with that for
template code generation AT compile time. However, there's a couple of BIG issues with that for
this particular code base.
1) Non-deterministic ordering, or more specifically for this app, no way to declare the
dependency on ***outputs*** of one generator as the ***input*** for another.
2) Dependencies for project references
- As a generator for this is not general purpose they would not be published or produced
as a NUGET package. They only would work as a project reference. But that creates a TON
- As a generator for this is not general purpose it would not be published or produced
as a NUGET package. It would only work as a project reference. But that creates a TON
of problems for the binary runtime dependencies of source generators, which don't flow
with them as project references...

Expand Down Expand Up @@ -114,15 +115,17 @@ how to generate the correct code.
significant factor.

#### The final choice
Keep using this library as a generator for the handle types. This used to work, and still does.
Keep using this app as a generator for the handle types. This used to work, and still does.
However, this doesn't solve the problem of expressing managed code things in a custom language
(YAML) but it's at least a rather simplistic expression for the handles. And arguably less
complicated then all the subtleties of using a Roslyn Source generator for this sort of one off
specialized code generation.

Solving the problem of expressing P/Invokes is simply to just manage that directly. It seemed
like a good idea to automate the tedium of generating those. Sadly, there are so many
subtleties that involve reading the docs (or source code) before you can correctly implement
it that there's no value in expressing all that subtlety in anything other than C#.
subtleties of "special cases" that involve reading the docs (or source code) before you can
correctly implement it. In the end, there's no value in expressing all that subtlety in anything
other than C#.

This also keeps the door open to use the native AST from within the source generator or an
analyzer to perform additional checks and ensure the hand written code matches the actual
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
// ------------------------------------------------------------------------------
// <auto-generated>
// This code was generated by a tool.
// Runtime Version: 18.0.0.0
// Runtime Version: 17.0.0.0
//
// Changes to this file may cause incorrect behavior and will be lost if
// the code is regenerated.
Expand All @@ -19,7 +19,7 @@ namespace LlvmBindingsGenerator.Templates
/// </summary>

#line 1 "D:\GitHub\Ubiquity.NET\Llvm.Net\src\Interop\LlvmBindingsGenerator\Templates\T4\WrappedHandleTemplate.tt"
[global::System.CodeDom.Compiler.GeneratedCodeAttribute("Microsoft.VisualStudio.TextTemplating", "18.0.0.0")]
[global::System.CodeDom.Compiler.GeneratedCodeAttribute("Microsoft.VisualStudio.TextTemplating", "17.0.0.0")]
internal partial class WrappedHandleTemplate : WrappedHandleTemplateBase
{
#line hidden
Expand Down Expand Up @@ -276,7 +276,7 @@ public void Dispose( )
/// <summary>
/// Base class for this transformation
/// </summary>
[global::System.CodeDom.Compiler.GeneratedCodeAttribute("Microsoft.VisualStudio.TextTemplating", "18.0.0.0")]
[global::System.CodeDom.Compiler.GeneratedCodeAttribute("Microsoft.VisualStudio.TextTemplating", "17.0.0.0")]
internal class WrappedHandleTemplateBase
{
#region Fields
Expand Down
32 changes: 17 additions & 15 deletions src/Interop/Ubiquity.NET.Llvm.Interop/ABI/ReadMe.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,7 @@ Input arrays are generally rather simple to declare and use the form:
void LLVMSomeApi(LLVMHandleOfSomeSort h, [In] UInt32[] elements, int numElements);
```

### Out arrays
### Arrays as OUT parameters
Arrays where the implementation is expected to provide a pointer that is allocated and
filled in by the native code use the following pattern:
``` C#
Expand Down Expand Up @@ -97,16 +97,18 @@ wrappers to avoid any potential confusion about the size of types between C# and
native C/C++)

### Distinction for real LLVMBOOL vs Status
This library and P/Invoke signatures disambiguates between an actual boolean value
This library and P/Invoke signatures disambiguate between an actual boolean value
(`LLVMBool`) and what is really a success or failure status code. As with strings, the
only way to tell the difference is to read the docs... Thus, for ALL API signatures an
LLVMStatus is used for any native code signature `LLVMBool` that is documented as
behaving like a status and NOT a bool. This prevents mass confusion on the intent and
helps keep the API surface cleaner and more self documenting. (Does a non-zero
`LLVMBool` mean it succeeded? Or does it mean there was an error?) Thus all APIs need
a developer to understand the documentation and set the P/Invoke signature to use
only way to tell the difference is to read the docs, or sometimes the source... Thus,
for ALL API signatures an `LLVMStatus` is used for any native code signature `LLVMBool`
that is documented as behaving like a status and NOT a bool. This prevents mass confusion
on the intent and helps keep the API surface cleaner and more self documenting. (Does a
non-zero `LLVMBool` mean it succeeded? Or does it mean there was an error?) Thus all APIs
need a developer to understand the documentation and set the P/Invoke signature to use
`LLVMStatus` for anything that is really a status code and ***NOT*** a proper boolean
(true/false) value.
(true/false) value. This library has done that work and ALL signatures using the native
`LLVMBool` is manually evaluated and the correct form applied to the managed P/Invoke
signature.

## Enumerated value naming
The normal rules of naming enumerated values are suspended for this low level interop
Expand All @@ -118,14 +120,14 @@ documentation for this library focuses ONLY on the interop support provided with
## Calling convention
All of the P/Invoke APIs use an explicit
`[UnmanagedCallConv(CallConvs=[typeof(CallConvCdecl)])]` to identify that the APIs use
the standard "C" ABI calling convention.
the standard "C" ABI calling convention. This is defined by the LLVM-C API.

### Future optimization
At some point in the future it might be worth adding the ability to suppress GC
transitions as an optimization. Application of that requires great care and
understanding of the GC and native implementation to ensure it is safe to do. This is
strictly a performance optimization that has NO impact on callers so is left for
future enhancement.
future enhancement as applicable.

## Special handling of strings
>[!NOTE]
Expand Down Expand Up @@ -209,9 +211,9 @@ private static unsafe partial uint LLVMGetSyncScopeID(LLVMContextRefAlias C, Laz

Sometimes an API will return a pointer AND include an out parameter for the length
of a string. These require great care as they do NOT guarantee that the string is
terminated! Only that the length is valid. Thus, such APIs follow a similar pattern
of wrapping that handles the out parameter to produce a `LazyEncodedString` as in the
following example:
terminated! Only that the length is valid. Thus, the implementation of wrappers for
such APIs follow a pattern of hiding the length out parameter to produce a
`LazyEncodedString` as in the following example:

``` C#
[MethodImpl(MethodImplOptions.AggressiveInlining)]
Expand Down Expand Up @@ -241,7 +243,7 @@ private static unsafe partial byte* LLVMGetStringAttributeKind(LLVMAttributeRef
> overflows are caught and triggered at the point of the problem. It is unlikely these
> will ever hit as strings that large are generally unmanageable at runtime anyway.


---
^1^ While it is plausible to create an ISO `C` compliant compiler implementation and
OS runtime environment where size_t is larger than a native pointer. Such a compiler
and runtime would have extremely limited use as most code written or found today,
Expand Down
23 changes: 16 additions & 7 deletions src/Interop/Ubiquity.NET.Llvm.Interop/ReadMe.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,16 +2,25 @@
This library contains the low level interop between managed code and the native library.
(Ubiquity.NET.LibLLVM).

>[!IMPORTANT]
> Consumers should ***NOT*** use this directly, instead they should use the [`Ubiquity.NET.Llvm`](https://www.nuget.org/packages/Ubiquity.NET.Llvm)
> package. That package includes a tested OO wrapper around the low level interop API. This
> package may not exist in the future as the functionality might fold into the OO wrapper
> library and any app built based on it would be stuck (on your own)
>
> :warning: You have been warned! :warning:

## Why not Direct P/Invoke?
[This is a thing: see some of the limited [docs](https://learn.microsoft.com/en-us/dotnet/core/deploying/native-aot/interop)
for more info]

Firstly it's young, poorly understood, barely even documented and not what is considered
by many as stable/mature yet. But, mostly because it requires everything to compile AOT to work. That
is, it's all or nothing and doesn't support use in an either or (traditional JIT runtime, or AOT).
Somethings are not yet supporting AOT/Trimming scenarios. (In the case of the samples in this repo
the DGML graph builder used employs XML serialization, which in turn requires dynamic reflection to
work. [So that has an AOT friendly replacement]) So, as great a feature AOT is it is not without
issues at present. Perhaps over time it will become the normal state and more libraries will build in
a way as to support it. (This library has a motivation to go there in that it's broader use is
specifically for AOT code generation! Thus it's a bit on the "bleeding edge" of things.)
is, it's all or nothing and doesn't support use in an either or model (traditional JIT runtime,
or AOT). Somethings are not yet supporting AOT/Trimming scenarios. (In the case of the samples
in this repo the DGML graph builder used employs XML serialization, which in turn requires
dynamic reflection to work. This required a custom solution to resolve. So, as great a feature
AOT is it is not without issues at present. Perhaps over time it will become the normal state
and more libraries will build in a way as to support it. (This library has a motivation to go
there in that it's broader use is specifically for AOT code generation! Thus it's a bit on the
"bleeding edge" of things.)
Original file line number Diff line number Diff line change
Expand Up @@ -14,14 +14,14 @@ public readonly record struct SymbolEntryInfo
/// <summary>Initializes a new instance of the <see cref="SymbolEntryInfo"/> struct.</summary>
/// <param name="name">Name of the symbol</param>
/// <param name="refCount">Reference count for the symbol</param>
public SymbolEntryInfo( string name, int refCount )
public SymbolEntryInfo( LazyEncodedString name, int refCount )
{
Name = name;
RefCount = refCount;
}

/// <summary>Gets the Name of the symbol</summary>
public string Name { get; }
public LazyEncodedString Name { get; }

public int RefCount { get; }
}
Expand Down
Loading
Loading