Marshal methods part 4 of 5 (runtime)

[grendello]

Context: dotnet/java-interop#1027

Implement the necessary runtime elements to enable running of applications with marshal methods.
This commit, if marshal methods are enabled, makes it possible to run both plain XA and MAUI apps.

The [UnmanagedCallersOnly] attribute used by marshal methods requires that the invoked method
doesn't have any non-blittable types as either its return type or type of any parameters passed
to the method. Implement generation of wrapper methods which replace bool (the only non-blittable
type currently found in any Mono.Android, AndroidX bindings or in MAUI) with byte and convert
the values appropriately before calling the actual target method. In a "hello world" MAUI test
application there are 133 such methods (out of around 180 total).

Implement code that enables us to show error messages with the proper assembly, class and method
names on failure to look up or obtain pointer to native callback methods.

[jonpryor]

Shouldn't this and other Console.WriteLine()s here use Log.LogDebugMessage() instead?

[grendello]

Not all of them will stay. The next PR in the series cleans that up.

[jonpryor]

I totally expected this method to "normalize" the empty string to null, not what this method actually does.

Also, given that this appears to be for "debug" output, why is it 'null' (with single-quotes)? Wouldn't outuput of:

method.CallbackField == null

make more sense than the current:

method.CallbackField == 'null'

?

[grendello]

This method will go away in the next PR in the series.

[jonpryor]

s/Console.WriteLine/Log.LogDebugMessage/g

[grendello]

Most of these will go away in the next PR

[jonpryor]

Is there actually need for many of these types to be public? We're not hitting them from a separate unit test assembly, so should they be public at all?

[grendello]

The next PR in the series will do all kinds of cleanups like this.

[jonpryor]

Is there an example of this? Given that the Java_… names contain the declaring package name and declaring type name, how can two different methods "from different classes" have the same name?

Or is "name" in this context the method name, not the Java_… name?

[grendello]

Yep, there are a few examples in the Hello Maui sample app:

Overloaded MM: Java_crc64e1fb321c08285b90_CellAdapter_n_1onActionItemClicked
  implemented in: Microsoft.Maui.Controls.Handlers.Compatibility.CellAdapter (System.Boolean Android.Views.ActionMode/ICallback::OnActionItemClicked(Android.Views.ActionMode,Android.Views.IMenuItem))
    mangling 'crc64e1fb321c08285b90/CellAdapter'
    mangling 'n_onActionItemClicked'
  Using FULL signature
    mangling 'Landroid/view/ActionMode;Landroid/view/MenuItem;'
     new native symbol name: Java_crc64e1fb321c08285b90_CellAdapter_n_1onActionItemClicked__Landroid_view_ActionMode_2Landroid_view_MenuItem_2
  implemented in: Microsoft.Maui.Controls.Handlers.Compatibility.CellAdapter (System.Boolean AndroidX.AppCompat.View.ActionMode/ICallback::OnActionItemClicked(AndroidX.AppCompat.View.ActionMode,Android.Views.IMenuItem))
    mangling 'crc64e1fb321c08285b90/CellAdapter'
    mangling 'n_onActionItemClicked'
  Using FULL signature
    mangling 'Landroidx/appcompat/view/ActionMode;Landroid/view/MenuItem;'
     new native symbol name: Java_crc64e1fb321c08285b90_CellAdapter_n_1onActionItemClicked__Landroidx_appcompat_view_ActionMode_2Landroid_view_MenuItem_2

Overloaded MM: Java_crc64e1fb321c08285b90_CellAdapter_n_1onCreateActionMode
  implemented in: Microsoft.Maui.Controls.Handlers.Compatibility.CellAdapter (System.Boolean Android.Views.ActionMode/ICallback::OnCreateActionMode(Android.Views.ActionMode,Android.Views.IMenu))
    mangling 'crc64e1fb321c08285b90/CellAdapter'
    mangling 'n_onCreateActionMode'
  Using FULL signature
    mangling 'Landroid/view/ActionMode;Landroid/view/Menu;'
     new native symbol name: Java_crc64e1fb321c08285b90_CellAdapter_n_1onCreateActionMode__Landroid_view_ActionMode_2Landroid_view_Menu_2
  implemented in: Microsoft.Maui.Controls.Handlers.Compatibility.CellAdapter (System.Boolean AndroidX.AppCompat.View.ActionMode/ICallback::OnCreateActionMode(AndroidX.AppCompat.View.ActionMode,Android.Views.IMenu))
    mangling 'crc64e1fb321c08285b90/CellAdapter'
    mangling 'n_onCreateActionMode'
  Using FULL signature
    mangling 'Landroidx/appcompat/view/ActionMode;Landroid/view/Menu;'
     new native symbol name: Java_crc64e1fb321c08285b90_CellAdapter_n_1onCreateActionMode__Landroidx_appcompat_view_ActionMode_2Landroid_view_Menu_2

Overloaded MM: Java_crc64e1fb321c08285b90_CellAdapter_n_1onDestroyActionMode
  implemented in: Microsoft.Maui.Controls.Handlers.Compatibility.CellAdapter (System.Void Android.Views.ActionMode/ICallback::OnDestroyActionMode(Android.Views.ActionMode))
    mangling 'crc64e1fb321c08285b90/CellAdapter'
    mangling 'n_onDestroyActionMode'
  Using FULL signature
    mangling 'Landroid/view/ActionMode;'
     new native symbol name: Java_crc64e1fb321c08285b90_CellAdapter_n_1onDestroyActionMode__Landroid_view_ActionMode_2
  implemented in: Microsoft.Maui.Controls.Handlers.Compatibility.CellAdapter (System.Void AndroidX.AppCompat.View.ActionMode/ICallback::OnDestroyActionMode(AndroidX.AppCompat.View.ActionMode))
    mangling 'crc64e1fb321c08285b90/CellAdapter'
    mangling 'n_onDestroyActionMode'
  Using FULL signature
    mangling 'Landroidx/appcompat/view/ActionMode;'
     new native symbol name: Java_crc64e1fb321c08285b90_CellAdapter_n_1onDestroyActionMode__Landroidx_appcompat_view_ActionMode_2

Overloaded MM: Java_crc64e1fb321c08285b90_CellAdapter_n_1onPrepareActionMode
  implemented in: Microsoft.Maui.Controls.Handlers.Compatibility.CellAdapter (System.Boolean Android.Views.ActionMode/ICallback::OnPrepareActionMode(Android.Views.ActionMode,Android.Views.IMenu))
    mangling 'crc64e1fb321c08285b90/CellAdapter'
    mangling 'n_onPrepareActionMode'
  Using FULL signature
    mangling 'Landroid/view/ActionMode;Landroid/view/Menu;'
     new native symbol name: Java_crc64e1fb321c08285b90_CellAdapter_n_1onPrepareActionMode__Landroid_view_ActionMode_2Landroid_view_Menu_2
  implemented in: Microsoft.Maui.Controls.Handlers.Compatibility.CellAdapter (System.Boolean AndroidX.AppCompat.View.ActionMode/ICallback::OnPrepareActionMode(AndroidX.AppCompat.View.ActionMode,Android.Views.IMenu))
    mangling 'crc64e1fb321c08285b90/CellAdapter'
    mangling 'n_onPrepareActionMode'
  Using FULL signature
    mangling 'Landroidx/appcompat/view/ActionMode;Landroid/view/Menu;'
     new native symbol name: Java_crc64e1fb321c08285b90_CellAdapter_n_1onPrepareActionMode__Landroidx_appcompat_view_ActionMode_2Landroid_view_Menu_2

[jonpryor]

Either you updated the comment, or I horribly misread this comment; I thought you were saying that the long/full name had conflicts/collisions.

What you're showing here are overloads.

For example, consider Activity.startActivity():

• Activity.startActivity(Intent)
• Activity.startActivity(Intent, Bundle)

You should be able to override both of these in C#:

class MyActivity : Activity {
    public override unsafe void StartActivity (Android.Content.Intent? intent) => …;
    public override unsafe void StartActivity (Android.Content.Intent? intent, Android.OS.Bundle? options) => …;
}

This would also result in needing to use the full Java_… name. (This would likely also make a good unit test in a future commit...)

Updating the comment to use the term "method overload" would be a helpful clarification. The fact that the methods being overridden came from different classes/interfaces is only slightly relevant; the more important bit is that within the JCW type, these are method overloads, a'la:

interface IntFooable {
    void foo(int value);
}
interface StringFooable {
    void foo(String value);
}
class MyFooable implements IntFooable, StringFooable {
    public native void foo(int value) {}
    public native void foo(String value) {}
}

Here, you can't just have Java_example_MyFooable_foo, because the method is overloaded. Instead, the full names must be used.

[grendello]

I modified the comment by adding the examples, but the rest of the wording remained the same. The "simple" overloads you give above are handled in the classifer, the case handled here is when two different managed types will end up having the same Java package name (like the "standard" Android vs AndroidX overloads in the comment) and thus become "native" overloads, requiring native symbol name with full signature.

[jonpryor]

The problem with this is that it'll "pass through" characters which shouldn't be passed through:

• https://discord.com/channels/732297728826277939/732297837953679412/1009499855674163231
• https://docs.oracle.com/javase/specs/jvms/se7/html/jvms-4.html#jvms-4.2.2

In particular, Kotlin methods can contain -. (We don't support overriding such methods yet, as we use Java as an intermediary, but if we ever start using Kotlin as an intermediary, we will be able to override such methods.)

Thus, what we should instead do is allow only [a-zA-Z0-9], and escape anything else:

if ((ch >= '0' && ch <= '0') ||
        (ch >= 'a' && ch <= 'z') ||
        (ch >= 'A' && ch <= 'Z')) {
    sb.Append (ch);
} else {
    sb.Append ("_0").Append (((int) ch).ToString ("x04"));
}

[grendello]

I don't know if restricting the allowed character set to [a-zA-Z0-9] will yield the same symbol name as computed by JNI. According to JNI docs, only non-ASCII characters are encoded in this way. I'd prefer to keep the current algorithm until we actually hit the Kotlin - problem.

[jonpryor]

Would make for a fascinating thing to test. Where's a good Java bytecode assembler? I need to create a Java method which contains \n!

[jonpryor]

Context: 903ba37ce70d2840983774e1d6fb55f8002561e2
Context: e1af9587bb98d4c249bbc392ceccc2b53ffff155

Context: https://github.com/xamarin/java.interop/issues/1027

Commit 903ba37c mentioned a TODO:
    
> Update/rewrite infrastructure to focus on implementing the runtime
> side of marshal methods, making it possible to actually run
> applications which use marshal methods.

Implement the necessary runtime elements to enable running of
applications with marshal methods.  It is now possible, if
LLVM marshal methods are enabled/`ENABLE_MARSHAL_METHODS` is defined,
to run both plain .NET SDK for Android and MAUI apps.

Update `src/Microsoft.Android.Sdk.ILLink/PreserveLists/System.Runtime.InteropServices.xml`
so that `System.Runtime.InteropServices.UnmanagedCallersOnlyAttribute`
is always preserved, as it is required by LLVM Marshal Methods.

The `[UnmanagedCallersOnly]` attribute used by marshal methods
requires that the invoked method have [blittable types][0] for the
method return type and all parameter types.  Unfortunately, `bool`
is *not* a blittable type.  Implement generation of wrapper methods
which replace `bool` with `byte` and convert the values appropriately
before calling the actual target method.  In a "hello world" MAUI
test app there are 133 such methods (out of around 180 total).

Implement code that enables us to show error messages with the proper
assembly, class and method names on failure to look up or obtain
pointer to native callback methods.

TODO:

  * Process *all* assemblies, including `Mono.Android.dll`, for
    Java Callable Wrapper generation.  This is necessary so that we
    can find and emit LLVM marshal methods for types defined within
    `Mono.Android.dll`.

  * Remove the `ENABLE_MARSHAL_METHODS` define, and enable
    LLVM marshal methods for everyone.

  * Update `<GenerateJavaStubs/>` to rewrite all assemblies for all
    Supported ABIs.  Currently, we don't support `Java.Lang.Object` &
    `Java.Lang.Throwable` subclasses being located in per-ABI
    assemblies.

  * How do `Java_…` native functions interact with
    `JNIEnv::RegisterNatives()`?
    https://github.com/xamarin/xamarin-android/pull/7285#discussion_r951755971

  * *Can* JNI `native` methods contain "non-printable" characters
    such as `\n`, or "non-representable in ELF symbols" characters
    such as `-` (e.g. Kotlin mangled methods)?
    https://github.com/xamarin/xamarin-android/pull/7285#discussion_r951789869

  * Cleanup, cleanup, cleanup

[0]: https://docs.microsoft.com/en-us/dotnet/framework/interop/blittable-and-non-blittable-types

[grendello]

The test failures in the latest run don't appear to be related to this PR which contains mostly disabled code. I'm going to merge, to make it possible to move on with marshal methods.