Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add basic support for frozen structs projections #2560

Merged
merged 21 commits into from
May 21, 2024
Merged
Show file tree
Hide file tree
Changes from 19 commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
1ebc0a9
Add basic support for frozen structs
kotlarmilos Apr 23, 2024
271f504
Refactor type database and registration system to handle dependencies
kotlarmilos Apr 30, 2024
d97c86f
Update tests to work with new namespace naming logic
kotlarmilos Apr 30, 2024
755c800
Update documentation to include details for frozen structs and type d…
kotlarmilos Apr 30, 2024
9f49861
Disable field record verification in StructEmitter
kotlarmilos Apr 30, 2024
c4b1c5c
Update test assertions for FrozenStructsTests
kotlarmilos Apr 30, 2024
97d2a98
Update test assertions for FrozenStructsTests
kotlarmilos Apr 30, 2024
4b8e403
Update runtime version
kotlarmilos Apr 30, 2024
7c00db6
Update .gitignore and sample app
kotlarmilos Apr 30, 2024
377ad48
first cut at factories
stephen-hawley May 14, 2024
f59bf96
Fix tooling options and introduce metadata accessor field
kotlarmilos May 16, 2024
bea3a01
Update sdk and arcade version
kotlarmilos May 16, 2024
9633191
Run field record metadata check in debug build only
kotlarmilos May 16, 2024
81a6117
Disable field record validation in StructEmitter
kotlarmilos May 16, 2024
6099dcd
Update test infrastructure to generate bindings and compile C# once p…
kotlarmilos May 16, 2024
2caf716
Add ArgumentDecl
kotlarmilos May 17, 2024
9c7a330
Introduce Conductor and Handler classes
kotlarmilos May 17, 2024
ccec89d
Remove redundant checks in type factories
kotlarmilos May 17, 2024
4924516
Refactor HandleBaseDecl method to handle list of base declarations
kotlarmilos May 17, 2024
8f6a52b
Update docs
kotlarmilos May 20, 2024
d4a8eed
Add license header to ArgumentDecl.cs
kotlarmilos May 21, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -201,4 +201,5 @@ launchSettings.json

# Testing artifacts
testing/
src/samples/**/*Bindings.cs
src/samples/**/Unsafe*.cs
src/samples/**/Swift.*.cs
69 changes: 57 additions & 12 deletions docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ This document provides a detailed overview of the .NET Swift interop tooling, fo

## Usage

The tooling consumes Swift ABI files, which are generated from `.swiftinterface` files by the Swift compiler. This ABI file contains a json representation of the abstract syntax tree of the `.swiftinterface` file. The `.swiftinterface` and `.abi.json` files are generated by executing the `swiftc` command with the `-emit-module-interface` option.
The tooling can consume a Swift ABI file or a framework name from the standard library. If a framework name is provided, the tool generates the ABI file based on the `.swiftinterface` file. This ABI file contains a JSON representation of the abstract syntax tree of the `.swiftinterface` file. Multiple Swift ABI files and frameworks can be specified for bindings.

```
Description:
Expand All @@ -14,15 +14,19 @@ Usage:
SwiftBindings [options]

Options:
-a, --swiftabi <swiftabi> (REQUIRED) Path to the Swift ABI file.
-o, --output <output> (REQUIRED) Output directory for generated bindings.
-v, --verbose <verbose> Prints information about work in process.
-h, --help Display a help message.
--version Show version information
-?, -h, --help Show help and usage information
-a, --swiftabi, -f, --framework Required. Path to the Swift ABI file or framework
-o, --output Required. Output directory for generated bindings
-platform Platform, e.g., MacOSX
-sdk SDK version, e.g., 14.4
-arch Architecture, e.g., arm64e
-target Target, e.g., apple-macos
-v Information about work in process
-h, --help Display a help message
--version Show version information
-?, -h, --help Show help and usage information
```

It is possible to specify multiple Swift ABI files for processing. If a unsupported type is encountered, the tooling will ignore it and generate C# source code for known syntax.
If an unsupported syntax element is encountered in the ABI file, the tooling will ignore it and generate C# source code for known syntax elements. The generated C# bindings are published as source files to the output directory, allowing users to modify them before compilation.

## Projections

Expand Down Expand Up @@ -136,6 +140,44 @@ namespace HelloLibraryBindings

In the example, the user's code references the `HelloLibraryBindings` namespace and invokes a static method that has the same name as the Swift function. When the Swift function returns a type, the C# wrapper method also returns type, with additional processing if required. The C# wrapper method is generated only when marshalling is required. If marshalling is not needed, only a P/Invoke declaration is generated.

### Frozen structs

Frozen structs that are POD or bitwise movable are projected as C# structs with sequential layout. Fields are retrieved from the ABI file and verified in debug builds against the runtime metadata:
```csharp
[StructLayout(LayoutKind.Sequential, Size = 14)]
public unsafe struct F0_S0 {
public Double f0;
public UInt32 f1;
public UInt16 f2;

[UnmanagedCallConv(CallConvs = new Type[] { typeof(CallConvSwift) })]
[DllImport("./libHelloLibrary.dylib", EntryPoint = "$s12HelloLibrary5F0_S0V2f02f12f2ACSd_s6UInt32Vs6UInt16VtcfC")]
internal static extern F0_S0 PIfunc_init(Double f0, UInt32 f1, UInt16 f2);
public F0_S0(Double f0, UInt32 f1, UInt16 f2)
{
this = PIfunc_init(f0, f1, f2);
}

[UnmanagedCallConv(CallConvs = new Type[] { typeof(CallConvSwift) })]
[DllImport("./libHelloLibrary.dylib", EntryPoint = "$s12HelloLibrary5F0_S0V9hashValueSiyF")]
internal static extern IntPtr PIfunc_hashValue(F0_S0 self);
public IntPtr hashValue()
{
F0_S0 self = this;
return PIfunc_hashValue(self);
}

}
```

## Type database

The type database contains all information related to the mapping between Swift and C#. The type registrar is a class used for handling modules and types, where types are organized into modules. Each Swift module in the database includes the C# namespace, path to the dynamic library, a flag indicating whether it has been processed, and a hash list of types. Each Swift type includes the C# namespace, a type identifier, and pointers to runtime metadata and the value witness table.

The type database serves as a central repository that manages all type mappings. Additionally, it collects dependencies required for a module and maintains lists of unprocessed types and modules.

This is the flow when a Swift type is encountered during ABI parsing: The `GetTypeMapping` function attempts to find type mappings between Swift and C#. It begins by searching for an existing mapping in the type database using the `Registrar.GetType` method with the provided Swift module and type name. If a mapping is found, it is returned. If not found, the function then tries to find the type within the `Swift.Runtime` namespace using the `Type.GetType` method. If the type is located, it is added to the database, marked as processed, and then returned. Otherwise, the type is registered in the database as unprocessed to allow for lazy-load processing within module dependencies. When a type is encountered during the ABI parsing, it is registered as processed in the database.

## Functional outline

The tooling comprises the following components:
Expand All @@ -146,10 +188,13 @@ The tooling comprises the following components:
- `emitter`: Emits a C# bindings library using string-based or object model-based emitter.
- **SwiftRuntime**: Library providing projections of common Swift types. It contains a type database for common Swift types and implements Swift runtime constructs in C#.

The general workflow for generating C# bindings from Swift code is as follows:
1. Consume the Swift ABI file (`.abi.json`) and aggregate the public ABI using a parser that generates module declarations.
2. If needed, generate marshalling information for collected ABI.
3. Generate C# source code using an emitter and generated declarations.
The general workflow for generating C# bindings from Swift code involves the following steps:
1. **Get the next ABI file or framework name:** The process begins by retrieving the next ABI path or framework name from the queue.
2. **(optional) Generate ABI file for frameworks:** If the input is a framework name, the Swift frontend compiler is utilized to generate the ABI file.
3. **Retrieve filters for ABI file:** After obtaining the ABI file, any type filters are retrieved to process the ABI file accordingly.
4. **Parse Swift ABI file:** The Swift ABI file is consumed, and the public ABI is aggregated as declarations using the ABI parser.
5. **Generate C# source code:** Using the aggregated declarations, C# source code is generated using the string-based emitter. This code generation process translates Swift declarations into equivalent C# representations.
6. **Queue unprocessed modules:** Finally, any unprocessed modules are pushed into the queue for subsequent processing cycles.

![Functional outline](functional-outline.svg)

Expand Down
4 changes: 2 additions & 2 deletions eng/Version.Details.xml
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
<?xml version="1.0" encoding="utf-8"?>
<Dependencies>
<ToolsetDependencies>
<Dependency Name="Microsoft.DotNet.Arcade.Sdk" Version="9.0.0-beta.24165.6">
<Dependency Name="Microsoft.DotNet.Arcade.Sdk" Version="9.0.0-beta.24260.2">
<Uri>https://github.com/dotnet/arcade</Uri>
<Sha>ace00d8719b8d1fdfd0cc05f71bb9af216338d27</Sha>
<Sha>480401b003bfd2eb989c315da5d6b99ad13a968c</Sha>
</Dependency>
</ToolsetDependencies>
</Dependencies>
2 changes: 1 addition & 1 deletion eng/Versions.props
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,6 @@
<XUnitVersion>2.4.1</XUnitVersion>
<XUnitRunnerVisualStudioVersion>2.4.3</XUnitRunnerVisualStudioVersion>
<!-- Set the custom NETCoreApp version -->
<MicrosoftNETCoreAppVersion>9.0.0-preview.3.24129.2</MicrosoftNETCoreAppVersion>
<MicrosoftNETCoreAppVersion>9.0.0-preview.3.24172.9</MicrosoftNETCoreAppVersion>
</PropertyGroup>
</Project>
6 changes: 3 additions & 3 deletions global.json
Original file line number Diff line number Diff line change
@@ -1,18 +1,18 @@
{
"sdk": {
"version": "9.0.100-preview.3.24153.2",
"version": "9.0.100-preview.3.24204.13",
"allowPrerelease": true,
"rollForward": "major"
},
"tools": {
"dotnet": "9.0.100-preview.3.24153.2",
"dotnet": "9.0.100-preview.3.24204.13",
"runtimes": {
"dotnet": [
"$(MicrosoftNETCoreAppVersion)"
]
}
},
"msbuild-sdks": {
"Microsoft.DotNet.Arcade.Sdk": "9.0.0-beta.24165.6"
"Microsoft.DotNet.Arcade.Sdk": "9.0.0-beta.24260.2"
}
}
26 changes: 0 additions & 26 deletions src/Swift.Bindings/src/Emitter/StringCSharpEmitter/ClassEmitter.cs

This file was deleted.

Original file line number Diff line number Diff line change
@@ -0,0 +1,223 @@
// Copyright (c) Microsoft Corporation.
// Licensed under the MIT License.

using System.CodeDom.Compiler;
using Swift.Runtime;

namespace BindingsGeneration
{
/// <summary>
/// Represents a method handler factory.
/// </summary>
public class MethodHandlerFactory : IFactory<BaseDecl, IMethodHandler>
{
/// <summary>
/// Checks if the factory can handle the declaration.
/// </summary>
/// <param name="decl">The base declaration.</param>
/// <returns></returns>
public bool Handles(BaseDecl decl)
{
return decl is MethodDecl;
}

/// <summary>
/// Constructs a handler.
/// </summary>
public IMethodHandler Construct ()
{
return new MethodHandler();
}
}

/// <summary>
/// Represents a method handler.
/// </summary>
public class MethodHandler : BaseHandler, IMethodHandler
{
public MethodHandler ()
{
}

/// <summary>
/// Marshals the method declaration.
/// </summary>
/// <param name="methodDecl">The method declaration.</param>
public IEnvironment Marshal(BaseDecl methodDecl)
{
return new MethodEnvironment(methodDecl);
}

/// <summary>
/// Emits the method declaration.
/// </summary>
/// <param name="writer">The IndentedTextWriter instance.</param>
/// <param name="env">The environment.</param>
/// <param name="conductor">The conductor instance.</param>
/// <param name="typeDatabase">The type database.</param>
public void Emit(IndentedTextWriter writer, IEnvironment env, Conductor conductor, TypeDatabase typeDatabase)
{
var methodEnv = (MethodEnvironment)env;
var methodDecl = (MethodDecl)methodEnv.MethodDecl;

// Emit PInvoke method
EmitPInvoke(writer, methodEnv, typeDatabase);

// Emit wrapper method if marshalling is required
if (methodDecl.ParentDecl is StructDecl || methodDecl.ParentDecl is ClassDecl)
{
if (methodDecl.IsConstructor)
{
// Emit constructor
EmitConstructor(writer, methodEnv);
}
else
{
// Emit method
EmitWrapperMethod(writer, methodEnv);
}
}

writer.WriteLine();
}

/// <summary>
/// Emits the PInvoke method declaration.
/// </summary>
/// <param name="writer">The IndentedTextWriter instance.</param>
/// <param name="env">The environment.</param>
/// <param name="typeDatabase">The type database.</param>
private void EmitPInvoke(IndentedTextWriter writer, MethodEnvironment env, TypeDatabase typeDatabase)
{
var methodDecl = (MethodDecl)env.MethodDecl;
var parentDecl = methodDecl.ParentDecl ?? throw new ArgumentNullException(nameof(methodDecl.ParentDecl));
var moduleDecl = methodDecl.ModuleDecl ?? throw new ArgumentNullException(nameof(methodDecl.ParentDecl));

string accessModifier = parentDecl == moduleDecl ? "public" : "internal";
string methodType = methodDecl.IsConstructor ? parentDecl.Name : methodDecl.Signature.First().TypeIdentifier.Name;
string methodName = parentDecl == moduleDecl ? methodDecl.Name : $"{env.PInvokePrefix}{methodDecl.Name}";
string libPath = typeDatabase.GetLibraryName(moduleDecl.Name);

writer.WriteLine("[UnmanagedCallConv(CallConvs = new Type[] { typeof(CallConvSwift) })]");
writer.WriteLine($"[DllImport(\"{libPath}\", EntryPoint = \"{methodDecl.MangledName}\")]");
writer.WriteLine($"{accessModifier} static extern {methodType} {methodName}({GetInternalMethodSignature(methodDecl)});");
}

/// <summary>
/// Emits the constructor declaration.
/// </summary>
/// <param name="writer">The IndentedTextWriter instance.</param>
/// <param name="env">The environment declaration.</param>
private void EmitConstructor(IndentedTextWriter writer, MethodEnvironment env)
{
var methodDecl = (MethodDecl)env.MethodDecl;
var parentDecl = methodDecl.ParentDecl ?? throw new ArgumentNullException(nameof(methodDecl.ParentDecl));
var moduleDecl = methodDecl.ModuleDecl ?? throw new ArgumentNullException(nameof(methodDecl.ParentDecl));

string methodName = $"{env.PInvokePrefix}{methodDecl.Name}";

writer.WriteLine($"public {parentDecl.Name}({GetPublicMethodSignature(methodDecl)})");
writer.WriteLine("{");
writer.Indent++;

string methodArgs = string.Join(", ", methodDecl.Signature.Skip(1).Select(p => p.Name));
writer.WriteLine($"this = {methodName}({GetMethodArgs(methodDecl)});");

writer.Indent--;
writer.WriteLine("}");
}

/// <summary>
/// Emits the wrapper method declaration.
/// </summary>
/// <param name="writer">The IndentedTextWriter instance.</param>
/// <param name="env">The environment.</param>
private void EmitWrapperMethod(IndentedTextWriter writer, MethodEnvironment env)
{
var methodDecl = (MethodDecl)env.MethodDecl;
var parentDecl = methodDecl.ParentDecl ?? throw new ArgumentNullException(nameof(methodDecl.ParentDecl));
var moduleDecl = methodDecl.ModuleDecl ?? throw new ArgumentNullException(nameof(methodDecl.ParentDecl));

string methodName = $"{env.PInvokePrefix}{methodDecl.Name}";

writer.WriteLine($"public {(methodDecl.MethodType == MethodType.Static ? "static " : "")}{methodDecl.Signature.First().TypeIdentifier.Name} {methodDecl.Name}({GetPublicMethodSignature(methodDecl)})");
writer.WriteLine("{");
writer.Indent++;

if (methodDecl.MethodType == MethodType.Instance)
{
writer.WriteLine($"{parentDecl.Name} self = this;");
}
string returnPrefix = methodDecl.Signature.First().TypeIdentifier.Name == "void" ? "" : "return ";
string methodArgs = string.Join(", ", methodDecl.Signature.Skip(1).Select(p => p.Name));
writer.WriteLine($"{returnPrefix}{methodName}({GetMethodArgs(methodDecl)});");
writer.Indent--;
writer.WriteLine("}");
}

/// <summary>
/// Gets the method parameters.
/// </summary>
/// <param name="methodDecl">The method declaration.</param>
/// <returns>The list of method parameters.</returns>
private List<ArgumentDecl> GetMethodParams(MethodDecl methodDecl)
{
var parentDecl = methodDecl.ParentDecl ?? throw new ArgumentNullException(nameof(methodDecl.ParentDecl));
List<ArgumentDecl> tempDecl = new(methodDecl.Signature);

// If this is a type method, add the marshalling for the self parameter
if (parentDecl is StructDecl || parentDecl is ClassDecl)
{
if (!methodDecl.IsConstructor && methodDecl.MethodType != MethodType.Static)
{
// Add self as the first parameter (after the return type)
tempDecl.Insert(1, new ArgumentDecl {
TypeIdentifier = new TypeDecl { Name = parentDecl.Name, MangledName = string.Empty, Fields = new List<FieldDecl>(), Declarations = new List<BaseDecl>(), ParentDecl = parentDecl, ModuleDecl = parentDecl.ModuleDecl},
Name = "self",
PrivateName = string.Empty,
IsInOut = false,
ParentDecl = methodDecl,
ModuleDecl = methodDecl.ModuleDecl
});
}
}

return tempDecl.Skip(1).ToList();
}

/// <summary>
/// Gets the internal method signature.
/// </summary>
/// <param name="moduleDecl">The module declaration.</param>
/// <returns>The internal method signature.</returns>
private string GetInternalMethodSignature(MethodDecl methodDecl)
{
var parentDecl = methodDecl.ParentDecl ?? throw new ArgumentNullException(nameof(methodDecl.ParentDecl));

List<ArgumentDecl> parameters = GetMethodParams(methodDecl);
return string.Join(", ", parameters.Select(p => $"{p.TypeIdentifier.Name} {p.Name}").ToList());
}

/// <summary>
/// Gets the public method signature.
/// </summary>
/// <param name="moduleDecl">The module declaration.</param>
/// <returns>The public method signature.</returns>
private string GetPublicMethodSignature(MethodDecl methodDecl)
{
List<ArgumentDecl> parameters = methodDecl.Signature.Skip(1).ToList();
return string.Join(", ", parameters.Select(p => $"{p.TypeIdentifier.Name} {p.Name}").ToList());
}

/// <summary>
/// Gets the method arguments.
/// </summary>
/// <param name="moduleDecl">The module declaration.</param>
/// <returns>The public method arguments.</returns>
private string GetMethodArgs(MethodDecl methodDecl)
{
List<ArgumentDecl> parameters = GetMethodParams(methodDecl);
return string.Join(", ", parameters.Select(p => p.Name).ToList());
}
}
}
Loading