Skip to content

Commit

Permalink
[Java.Base-Tests] Test Java-to-Managed invocations for Java.Base (#975)
Browse files Browse the repository at this point in the history
Commit bc5bcf4 (Desktop `Java.Base` binding) had a TODO:

> ~~ TODO: Marshal Methods ~~
>
> Marshal methods are currently skipped.  Java-to-managed invocations
> are not currently supported.

No marshal methods means no facility for Java-to-managed invocations.

Add a new `tests/Java.Base-Tests` unit test assembly, and test
Java-to-managed invocations.

This requires touching and updating ~everything. 😅

Update `Java.Interop.dll` to contain a new
`Java.Interop.JniMethodSignatureAttribute` custom attribute.

Update `generator` to begin emitting `[JniMethodSignature]` on bound
methods.  This is necessary so that `jcw-gen` knows the JNI method
signature of Java methods to emit.  As part of this, *remove* the
`JniTypeSignatureAttr` type added in bc5bcf4; trying to follow the
`JniTypeSignatureAttr` pattern for `JniMethodSignatureAttr` would
make for more intrusive code changes.  Instead, "re-use" the existing
`RegisterAttr` type, adding a `RegisterAttr.MemberType` property so
that we can select between Android `[Register]` output vs. "Desktop"
`[JniTypeSignature]` and `[JniMethodSignature]` output.  This in turn
requires updating many `generator-Tests` artifacts.

Update `Java.Interop.Tools.JavaCallableWrappers` to look for
`Java.Interop.JniTypeSignatureAttribute` and
`Java.Interop.JniMethodSignatureAttribute`.  This allows
`jcw-gen Java.Base-Tests.dll` to generate Java Callable Wrappers for
`Java.BaseTests.MyRunnable`.

Update `Java.Interop.Export.dll` so that `MarshalMemberBuilder`
doesn't use `Expression.GetActionType()` or
`Expression.GetFuncType()`, as .NET doesn't support the use of
generic types in [`Marshal.GetFunctionPointerForDelegate()`][0].
Instead, we need to use `System.Reflection.Emit` to define our own
custom delegate types.

~~ Comparison with Android ~~

Android bindings (both Classic Xamarin.Android and
.NET SDK for Android) use `generator`-emitted "connector methods"
which are specified in the `[Register]` attribute (see also 99897b2):

	namespace Java.Lang {
	    [Register ("java/lang/Object" DoNotGenerateAcw=true)]
	    partial class Object {
	        [Register ("equals", "(Ljava/lang/Object;)Z", "GetEquals_Ljava_lang_Object_Handler")]
	        public virtual unsafe bool Equals (Java.Lang.Object? obj)
	            => …

	        static Delegate GetEquals_Ljava_lang_Object_Handler()
	            => JNINativeWrapper.CreateDelegate((_JniMarshal_PPL_Z) n_Equals_Ljava_lang_Object_);
	        static bool n_Equals_Ljava_lang_Object_ (IntPtr jnienv, IntPtr native__this, IntPtr native_obj)
	            => …
	    }
	}

`jcw-gen` emits the "connector method" into the resulting
Java Callable Wrappers for appropriate subclasses:

	String __md_methods =
	    "n_equals:(Ljava/lang/Object;)Z:GetEquals_Ljava_lang_Object_Handler\n";

At runtime, `AndroidTypeManager.RegisterNativeMembers()` will lookup
`Object.GetEquals_Ljava_lang_Object_Handler()` and invoke it.
The returned `Delegate` instance is provided to
`JNIEnv::RegisterNatives()`.

The problem with this approach is that it's inflexible: there is no
way to participate in the "connector method" infrastructure to alter
marshaling behavior.  If you need custom behavior, e.g.
`Android.Graphics.Color` customizations, you need to update
`generator` and rebuild the binding assemblies.

(This has been "fine" for the past 10+ years, so this hasn't been a
deal breaker.)

For `Java.Base`, @jonpryor wants to support the custom marshaling
infrastructure introduced in 77a6bf8.  This would allow types to
participate in JNI marshal method ("connector method") generation
*at runtime*, allowing specialization based on the current set of
types and assemblies.

This means we *don't* need to specify a "connector method" in
`[JniMethodSignatureAttribute]`, nor generate them.

	namespace Java.Lang {
	    [JniTypeSignatureAttribute("java/lang/Object", GenerateJavaPeer=false)]
	    partial class Object {
	        [JniMethodSignatureAttribute("equals", "(Ljava/lang/Object;)Z")]
	        public virtual unsafe bool Equals (Java.Lang.Object? obj)
	            => …

	        // No GetEquals_Ljava_lang_Object_Handler()
	        // No n_Equals_Ljava_lang_Object_()
	    }
	}

This should result in smaller binding assemblies.

The downside is that runtime costs *increase*, significantly.
Instead of looking up and invoking a "connector method", the method
must be *generated* via System.Linq.Expressions expression trees,
then compiled into IL, then JIT'd, all before it can be used.

We have no timing data to indicate how "bad" this overhead will be.

As always, the hope is that `tools/jnimarshalmethod-gen` (176240d)
can be used to get the "best of both worlds": the flexibility of
custom marshalers, without the runtime overhead.  But…

  * #14
  * #616

At this point in time, `jnimarshalmethod-gen` *cannot* work under
.NET 6.  This will need to be addressed in order for custom marshalers
to be a useful solution.

`Java.Base` will use "connector method"-less bindings to act as a
"forcing function" in getting `jnimarshalmethod-gen` working in .NET.

[0]: https://docs.microsoft.com/en-us/dotnet/api/system.runtime.interopservices.marshal.getfunctionpointerfordelegate?view=net-6.0
  • Loading branch information
jonpryor committed May 5, 2022
1 parent 5971625 commit 4787e01
Show file tree
Hide file tree
Showing 69 changed files with 2,464 additions and 201 deletions.
7 changes: 7 additions & 0 deletions Java.Interop.sln
Original file line number Diff line number Diff line change
Expand Up @@ -105,6 +105,8 @@ Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Java.Interop.Tools.JavaType
EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Java.Base", "src\Java.Base\Java.Base.csproj", "{30DCECA5-16FD-4FD0-883C-E5E83B11565D}"
EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Java.Base-Tests", "tests\Java.Base-Tests\Java.Base-Tests.csproj", "{CB05E11B-B96F-4179-A4E9-5D6BDE29A8FC}"
EndProject
Global
GlobalSection(SharedMSBuildProjectFiles) = preSolution
src\Java.Interop.NamingCustomAttributes\Java.Interop.NamingCustomAttributes.projitems*{58b564a1-570d-4da2-b02d-25bddb1a9f4f}*SharedItemsImports = 5
Expand Down Expand Up @@ -296,6 +298,10 @@ Global
{30DCECA5-16FD-4FD0-883C-E5E83B11565D}.Debug|Any CPU.Build.0 = Debug|Any CPU
{30DCECA5-16FD-4FD0-883C-E5E83B11565D}.Release|Any CPU.ActiveCfg = Release|Any CPU
{30DCECA5-16FD-4FD0-883C-E5E83B11565D}.Release|Any CPU.Build.0 = Release|Any CPU
{CB05E11B-B96F-4179-A4E9-5D6BDE29A8FC}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{CB05E11B-B96F-4179-A4E9-5D6BDE29A8FC}.Debug|Any CPU.Build.0 = Debug|Any CPU
{CB05E11B-B96F-4179-A4E9-5D6BDE29A8FC}.Release|Any CPU.ActiveCfg = Release|Any CPU
{CB05E11B-B96F-4179-A4E9-5D6BDE29A8FC}.Release|Any CPU.Build.0 = Release|Any CPU
EndGlobalSection
GlobalSection(SolutionProperties) = preSolution
HideSolutionNode = FALSE
Expand Down Expand Up @@ -346,6 +352,7 @@ Global
{B173F53B-986C-4E0D-881C-063BBB116E1D} = {0998E45F-8BCE-4791-A944-962CD54E2D80}
{11942DE9-AEC2-4B95-87AB-CA707C37643D} = {271C9F30-F679-4793-942B-0D9527CB3E2F}
{30DCECA5-16FD-4FD0-883C-E5E83B11565D} = {0998E45F-8BCE-4791-A944-962CD54E2D80}
{CB05E11B-B96F-4179-A4E9-5D6BDE29A8FC} = {271C9F30-F679-4793-942B-0D9527CB3E2F}
EndGlobalSection
GlobalSection(ExtensibilityGlobals) = postSolution
SolutionGuid = {29204E0C-382A-49A0-A814-AD7FBF9774A5}
Expand Down
9 changes: 9 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,9 @@ TESTS = \
bin/Test$(CONFIGURATION)/Java.Interop.Tools.Generator-Tests.dll \
bin/Test$(CONFIGURATION)/Xamarin.SourceWriter-Tests.dll

NET_TESTS = \
bin/Test$(CONFIGURATION)-net6.0/Java.Base-Tests.dll

PTESTS = \
bin/Test$(CONFIGURATION)/Java.Interop-PerformanceTests.dll

Expand All @@ -44,6 +47,7 @@ run-all-tests:
r=0; \
$(MAKE) run-tests || r=1 ; \
$(MAKE) run-test-jnimarshal || r=1 ; \
$(MAKE) run-net-tests || r=1 ; \
$(MAKE) run-ptests || r=1 ; \
$(MAKE) run-java-source-utils-tests || r=1 ; \
exit $$r;
Expand Down Expand Up @@ -123,6 +127,11 @@ run-tests: $(TESTS) bin/Test$(CONFIGURATION)/$(JAVA_INTEROP_LIB)
$(foreach t,$(TESTS), $(call RUN_TEST,$(t),1)) \
exit $$r;

run-net-tests: $(NET_TESTS) bin/Test$(CONFIGURATION)-net6.0/$(JAVA_INTEROP_LIB)
r=0; \
$(foreach t,$(NET_TESTS), dotnet test $(t) || r=1) \
exit $$r;

run-ptests: $(PTESTS) bin/Test$(CONFIGURATION)/$(JAVA_INTEROP_LIB)
r=0; \
$(foreach t,$(PTESTS), $(call RUN_TEST,$(t))) \
Expand Down
9 changes: 9 additions & 0 deletions build-tools/automation/templates/core-tests.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -130,6 +130,15 @@ steps:
arguments: bin/Test$(Build.Configuration)$(NetCoreTargetFrameworkPathSuffix)/Java.Interop-PerformanceTests.dll
continueOnError: true

- task: DotNetCoreCLI@2
displayName: 'Tests: Java.Base'
condition: or(eq('${{ parameters.runNativeDotnetTests }}', 'true'), eq('${{ parameters.runNativeTests }}', 'true'))
inputs:
command: test
testRunTitle: Java.Base (net6.0 - ${{ parameters.platformName }})
arguments: bin/Test$(Build.Configuration)$(NetCoreTargetFrameworkPathSuffix)/Java.Base-Tests.dll
continueOnError: true

- task: DotNetCoreCLI@2
displayName: 'Tests: java-source-utils'
inputs:
Expand Down
1,695 changes: 1,664 additions & 31 deletions src/Java.Base-ref.cs

Large diffs are not rendered by default.

57 changes: 48 additions & 9 deletions src/Java.Interop.Export/Java.Interop/MarshalMemberBuilder.cs
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@
using System.Linq;
using System.Linq.Expressions;
using System.Reflection;
using System.Reflection.Emit;
using System.Text;

using Java.Interop.Expressions;
Expand Down Expand Up @@ -241,15 +242,6 @@ public LambdaExpression CreateMarshalToManagedExpression (MethodInfo method, Jav

static Type GetMarshalerType (Type returnType, List<Type> funcTypeParams, Type declaringType)
{
// `mscorlib.dll` & `System.Core.dll` only provide Action<…>/Func<…> types for up to 16 parameters
if (funcTypeParams.Count <= 16) {
if (returnType != null)
funcTypeParams.Add (returnType);
return returnType == null
? Expression.GetActionType (funcTypeParams.ToArray ())
: Expression.GetFuncType (funcTypeParams.ToArray ());
}

// Too many parameters; does a `_JniMarshal_*` type exist in the type's declaring assembly?
funcTypeParams.RemoveRange (0, 2);
var marshalDelegateName = new StringBuilder ();
Expand All @@ -265,11 +257,58 @@ static Type GetMarshalerType (Type returnType, List<Type> funcTypeParams, Type d
}

Type marshalDelegateType = declaringType.Assembly.GetType (marshalDelegateName.ToString (), throwOnError: false);
if (marshalDelegateType != null) {
return marshalDelegateType;
}

#if !NET
// Punt?; System.Linq.Expressions will automagically produce the needed delegate type.
// Unfortunately, this won't work with jnimarshalmethod-gen.exe.
return marshalDelegateType;
#else // NET
return CreateMarshalDelegateType (marshalDelegateName.ToString (), returnType, funcTypeParams);
#endif // NET
}

#if NET
static object ab_lock = new object ();
static AssemblyBuilder assemblyBuilder;
static ModuleBuilder moduleBuilder;
static Type[] DelegateCtorSignature;

static Type CreateMarshalDelegateType (string name, Type returnType, List<Type> funcTypeParams)
{
lock (ab_lock) {
if (assemblyBuilder == null) {
var aname = new AssemblyName ("jni-marshal-method-delegates");
assemblyBuilder = AssemblyBuilder.DefineDynamicAssembly (aname, AssemblyBuilderAccess.Run);
moduleBuilder = assemblyBuilder.DefineDynamicModule (aname.Name!);

DelegateCtorSignature = new Type[] {
typeof (object),
typeof (IntPtr)
};
}
funcTypeParams.Insert (0, typeof (IntPtr));
funcTypeParams.Insert (0, typeof (IntPtr));
var typeBuilder = moduleBuilder.DefineType (
name,
TypeAttributes.Class | TypeAttributes.Public | TypeAttributes.Sealed | TypeAttributes.AnsiClass | TypeAttributes.AutoClass,
typeof (MulticastDelegate)
);

const MethodAttributes CtorAttributes = MethodAttributes.RTSpecialName | MethodAttributes.HideBySig | MethodAttributes.Public;
const MethodImplAttributes ImplAttributes = MethodImplAttributes.Runtime | MethodImplAttributes.Managed;
const MethodAttributes InvokeAttributes = MethodAttributes.Public | MethodAttributes.HideBySig | MethodAttributes.NewSlot | MethodAttributes.Virtual;

typeBuilder.DefineConstructor (CtorAttributes, CallingConventions.Standard, DelegateCtorSignature)
.SetImplementationFlags (ImplAttributes);
typeBuilder.DefineMethod ("Invoke", InvokeAttributes, returnType, funcTypeParams.ToArray ())
.SetImplementationFlags (ImplAttributes);
return typeBuilder.CreateTypeInfo ();
}
}
#endif // NET

static char GetJniMarshalDelegateParameterIdentifier (Type type)
{
Expand Down
Loading

0 comments on commit 4787e01

Please sign in to comment.