Implement projection tooling for simple P/Invoke cases

[kotlarmilos]

Most of the code has been moved from https://github.com/xamarin/binding-tools-for-swift.

This PR is quite large, and there are multiple options for reviewing it. It includes all necessary components to demonstrate the end-to-end flow and allow for CI testing.

I have attempted to prune the tooling as much as possible, though there are some limitations. When reviewing this PR, I recommend initially focusing on the tooling UX and the quality of the generated bindings rather than the internal workings of the tooling.

Here are my suggestions for the review process: Begin with the SwiftRuntimeLibrary, it is a small project that outlines the implementation of Swift types in C#. The SwiftBindings project includes the tool interface and compilers for bindings. The SwiftReflector project includes ABI aggregation logic, module declarations, and type system. The SyntaxDynamo project includes API for C# source code generation and can potentially be improved at a later stage as it shouldn't impact the quality of the generated bindings.

Description

This PR represents a minimal set of features required for a simple direct P/Invoke scenario. It includes the following components, along with unit tests and samples.

• SwiftBindings: Command-line interface that orchestrates the tooling workflow.
• SwiftReflector: A library provides support for public ABI aggregation. It contains implementation of module declarations and the type system.
• SwiftRuntimeLibrary: Library that provides runtime marshaling support and projections of common Swift types.
• SyntaxDynamo: Library that provides an API for generating C# source code.

Usage

Options:

Description:
  Swift bindings generator.

Usage:
  SwiftBindings [options]

Options:
  -a, --swiftabi <swiftabi> (REQUIRED)  Path to the Swift ABI file.
  -o, --output <output> (REQUIRED)      Output directory for generated bindings.
  -v, --verbose <verbose>               Prints information about work in process.
  -h, --help                            Display a help message.
  --version                             Show version information
  -?, -h, --help                        Show help and usage information

Example usage:

dotnet SwiftBindings.dll -a "./HelloLibrary.abi.json" -o "./"

If a unsupported type is encountered, the tooling will ignore it and generate C# source code for known syntax.

Simple binding

The supported scenario involves simple P/Invoke calls without primitive params and no exceptions.

Swift library example:

public func sayHello() {
    print("Hello world")
}

User's code in C#:

using System;
using HelloLibraryBindings;

namespace HelloWorld
{
    public class Program
    {
        public static void Main(string[] args)
        {
            HelloLibrary.sayHello();
        }
    }
}

Generated bindings in C#:

using System;
using System.Runtime.InteropServices;
namespace HelloLibraryBindings
{
    public class HelloLibrary
    {
        [DllImport("libHelloLibrary.dylib", EntryPoint = "$s12HelloLibrary03sayA0yyF")]
        internal static extern void PIfunc_sayHello();
        public static void sayHello()
        {
            PIfunc_sayHello();
        }
    }
}

The design discussion for the simple P/Invoke scenarios is available at #2518. The documentation is available at #2512.

[jkoritzinsky]

Some basic comments around the sample/infra to start.

I'm a little concerned by the fact that we have a totally independent C# syntax writer here that also seems to understand Swift syntax, not just SwiftInterface syntax (likely from when BTfS would write separately). Maybe we can delete the Swift-writing support or move it into a separate assembly now that that's not on the roadmap?

[jkoritzinsky]

Let's use the Swift language support in CMake and require using a generator that supports it (ie. Ninja). As this code isn't going in the runtime repo, we can be more reasonable about requiring specific features/options.

Suggested change

	project(SmokeTests)
	project(SmokeTests Swift)

[kotlarmilos]

It seems that the ninja generator is not installed on build machines: https://github.com/actions/runner-images/blob/macOS-12/20240202.1/images/macos/macos-12-Readme.md

[jkotas]

https://github.com/actions/runner-images/blob/macOS-12/20240202.1/images/macos/macos-12-Readme.md

Nit: We use our build images. This link is for GitHub Actions images - I do not think it is relevant to our builds.

[stephen-hawley]

The original design was to operate on multiple libraries, but frankly, this made things worse overall. I feel like we should simplify and make it take only a single dylib/swiftinterface pair.

[stephen-hawley]

As we discussed in the meeting, I think it's reasonable to shed everything before swift 5 and throw if we get anything else.

[kotlarmilos]

Thanks, removed. Is it safe to remove the kSwift4xID? It is used in

f (swiftIdentifier.StartsWith (kSwift4xID, StringComparison.Ordinal)|| swiftIdentifier.StartsWith (kSwift5ID, StringComparison.Ordinal)) {

[stephen-hawley]

__T and _$s are legacy

[kotlarmilos]

According to the ABI document, the _$s is used for globals: https://github.com/apple/swift/blob/main/docs/ABI/Mangling.rst#L11

Did you mean _$S perhaps?

[jkotas]

https://github.com/dotnet/arcade/blob/main/Documentation/MirroringPackages.md is the way to do it correctly.

[kotlarmilos]

Maybe we can delete the Swift-writing support or move it into a separate assembly now that that's not on the roadmap?

I agree, let's try to narrow the scope at this point.

Generated code shouldn't get checked in. The original BTfS repository has tooling to get the appropriate version of antlr and generate the parser as well as debugging code for it.

I've implemented (and will upstream) a prototype ABI parser that consumes .abi.json instead of .swiftinterface and .dylib. The prototype creates module declarations and relies on the underlying type system. It removes dependencies on Demangling, Inventory, SwiftInterfaceParser, and Antlr4.Runtime.Standard (Java). Additionally, it should simplify the UX.

[rolfbjarne]

If a unsupported type is encountered, the tooling will ignore it and generate C# source code for known syntax.

So one test scenario would be to throw all the .swiftinterface files Xcode ships at it and verify that the binding process doesn't fail.

for sw in $(find /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/System/Library/Frameworks -name '*.swiftinterface'); do
    dotnet SwiftBindings.dll --swiftinterface $sw ...
done

[kotlarmilos]

The scope of the tooling has been narrowed to only support projecting public functions into static C# methods with P/Invoke declarations. The tooling has been split into two phases: public ABI aggregation and code generation. The integration point is ISwiftParser and ModuleDeclaration as implemented in https://github.com/dotnet/runtimelab/blob/swift-bindings/basic-tooling/src/SwiftBindings/src/Program.cs#L69-L86.

For simplicity, the Swift interface parser has been temporarily removed and will be introduced in subsequent PRs. Stress tests have been added to https://github.com/dotnet/runtimelab/tree/swift-bindings/basic-tooling/src/SwiftBindings/tests (kudos to @jakobbotsch).

Please reflect on important points that you think should be addressed in this PR, with focus on the SwiftBindings project.

[kotlarmilos]

So one test scenario would be to throw all the .swiftinterface files Xcode ships at it and verify that the binding process doesn't fail.

Good idea. I suggest to bundle it with .swiftinterface parser changes.

[kotlarmilos]

To facilitate easier code review, this PR will be divided into smaller PRs:

• Swift bindings generator with simple parser and emitter #2525 that provides runtime marshaling support and projections of common Swift types provides an MVP implementation of bindings generator with simplified parser and emitter.
  - [ ] Add SyntaxDynamo project #2526 that provides an API for generating C# source code
  - [ ] SwiftReflector project that provides support for public ABI aggregation
  - [ ] SwiftBindings project that provides a command-line interface

[kotlarmilos]

After discussion in #2526, we are closing this PR in favor of an MVP proposed in #2525.