Marshal methods, Part 1 of ?

[grendello]

This PR brings the lowest level "plumbing" for the future marshal methods implementation.

The code is disabled because it's not ready to work for anyone else except me, but I'd
like to split the code up into chunks that are easier to review, as the entire thing
is likely to be very big.

The code just needs to compile and stay out of the way.

Marshal methods are a feature designed to replace the current mechanism of "registering"
managed types with the Java runtime (via JNI). The way this is currently done, is the
JI binding generator generates 3 managed methods per single overridable Java class member,
for instance Android.App.Activity::OnCreate:

static Delegate? cb_onCreate_Landroid_os_Bundle_;
static Delegate GetOnCreate_Landroid_os_Bundle_Handler ()
{
  if (cb_onCreate_Landroid_os_Bundle_ == null)
    cb_onCreate_Landroid_os_Bundle_ = JNINativeWrapper.CreateDelegate ((_JniMarshal_PPL_V) n_OnCreate_Landroid_os_Bundle_);
  return cb_onCreate_Landroid_os_Bundle_;
}

static void n_OnCreate_Landroid_os_Bundle_ (IntPtr jnienv, IntPtr native__this, IntPtr native_savedInstanceState)
{
  var __this = global::Java.Lang.Object.GetObject<Android.App.Activity> (jnienv, native__this, JniHandleOwnership.DoNotTransfer)!;
  var savedInstanceState = global::Java.Lang.Object.GetObject<Android.OS.Bundle> (native_savedInstanceState, JniHandleOwnership.DoNotTransfer);
  __this.OnCreate (savedInstanceState);
}

// Metadata.xml XPath method reference: path="/api/package[@name='android.app']/class[@name='Activity']/method[@name='onCreate' and count(parameter)=1 and parameter[1][@type='android.os.Bundle']]"
[Register ("onCreate", "(Landroid/os/Bundle;)V", "GetOnCreate_Landroid_os_Bundle_Handler")]
protected virtual unsafe void OnCreate (Android.OS.Bundle? savedInstanceState)
{
  const string __id = "onCreate.(Landroid/os/Bundle;)V";
  try {
    JniArgumentValue* __args = stackalloc JniArgumentValue [1];
    __args [0] = new JniArgumentValue ((savedInstanceState == null) ? IntPtr.Zero : ((global::Java.Lang.Object) savedInstanceState).Handle);
    _members.InstanceMethods.InvokeVirtualVoidMethod (__id, this, __args);
  } finally {
    global::System.GC.KeepAlive (savedInstanceState);
  }
}

Another generator, JavaCallableWrappersGenerator, is used to output Java code whenever the application
overrides any of the Java members, e.g. for the same OnCreate above:

public static final String __md_methods;
static {
  __md_methods = 
    "n_onCreate:(Landroid/os/Bundle;)V:GetOnCreate_Landroid_os_Bundle_Handler\n" +
    "";
    mono.android.Runtime.register ("HelloAndroid.MainActivity, HelloAndroid", MainActivity.class, __md_methods);
}

public void onCreate (android.os.Bundle p0)
{
  n_onCreate (p0);
}

private native void n_onCreate (android.os.Bundle p0);

From there, the mono.android.Runtime.register call is a function defined in the native Xamarin.Android runtime
which takes the method descriptions and, when called by Java, passes them to the managed land which iterates
over the __md_methods array, processing each string to register (using JNI) the method with Java runtime.

Marshal methods will replace the registration mechanism, thus removing the .register call and __md_methods
from the Java side, instead generating a set of native functions in the Xamarin.Android runtime with correct symbol
names that will allow the Java runtime to call the, e.g., n_OnCreate above, inside our native code. This will
remove the need for entire machinery that currently emits CIL code at the managed registration stage.

It will also allow us to remove the delegate backing field and the GetOnCreate_Landroid_os_Bundle_Handler method (and
its counterparts for other overrides) from the generated C# code. The way this is going to work is that the
handler method (n_OnCreate_Landroid_os_Bundle_ above) will be decorated with the [UnmanagedCallersOnly] attribute
which will allow it to be invoked directly from the generated native function (the n_OnCreate implementation in the
example above).

The current plan of how the above modifications are going to work at build time is that the JI C# code generator
will output two versions of the native handler code (the n_OnCreate_Landroid_os_Bundle_ above) where one of them
will have the [UnmanagedCallersOnly] attribute and the other will not. The function decorated with the attribute will
have a slightly different name (since it will otherwise have the same signature), e.g. n_OnCreate_Landroid_os_Bundle_MM.
Then, when the application is built and marshal methods are enabled, the linker will remove the static backing field which
stores the delegate, as well as the GetOnCreate_Landroid_os_Bundle_Handler method, together with the
n_OnCreate_Landroid_os_Bundle_ method. If marshal methods are disabled, however, only the n_OnCreate_Landroid_os_Bundle_MM
will be removed.

[jonathanpeppers]

Is building with dotnet build a requirement for this one? When you build inside Visual Studio, it will be Full using .NET Framework MSBuild.

[grendello]

Yep, marshal methods will be NET7+ only

[jonathanpeppers]

Well, but you can build .NET 7 projects in Visual Studio with .NET Framework MSBuild.

Does this feature require our build to be running under .NET 7?

[jonathanpeppers]

We can maybe shell out to dotnet to make this work, if needed.

[grendello]

Native portion of the feature uses APIs that aren't available in Mono classic, so I wanted to make sure it's not enabled for classic by accident. However, this properly is here just until I finish all the parts of the feature, it will be
removed in the last PR. It's just easier this way for me to keep the feature disabled in main while I can enable it globally in the branch I'm working in on the next PRs.

[jonpryor]

@grendello: please provide a description of the proposed architecture, how all the pieces fit together, and what the resulting output "looks like" conceptually.

[grendello]

@grendello: please provide a description of the proposed architecture, how all the pieces fit together, and what the resulting output "looks like" conceptually.

I can provide a bird's eye view of the architecture, I don't know exactly how the details are going to look. I'd rather leave writing docs till the last commit in the series. Unless you want that high altitude overview?

[jonpryor]

I know large fractions of this code aren't even built in this PR, but why have this enum when only AssemblyLoadContext will be supported for marshal methods?

[grendello]

It's a different kind of context, I'm not even sure I'm going to use this enum. You can ignore it for now :)

[jonpryor]

Silly idea? store that "friendly string" as part of the struct. Alas, that'll make things bigger…

[grendello]

Yeah, I thought about that, it might be the best idea

[jonpryor]

Seems odd that libmonodroid.so is ~26KB larger, yet lots of the new C++ code is not even compiled. Weird.

[grendello]

A lot more code is inlined now. It's a bit surprising it generates that much of a difference, tbh.

[jonpryor]

[Xamarin.Android.Build.Tasks, monodroid] LLVM Marshal Methods Infra (#7004)

Context: https://github.com/xamarin/xamarin-android/wiki/Blueprint#java-type-registration

Introduce low-level "plumbing" for future JNI marshal method work.

A JNI marshal method is a [JNI-callable][0] function pointer provided
to [`JNIEnv::RegisterNatives()`][1].  Currently, JNI marshal methods
are provided via the interaction between `generator` and
`JNINativeWrapper.CreateDelegate()`:

  * `generator` emits the "actual" JNI-callable method.
  * `JNINativeWrapper.CreateDelegate()` uses System.Reflection.Emit
    to *wrap* the `generator`-emitted for exception marshaling.
    (Though see also 32cff438.)

JNI marshal methods are needed for all Java-to-C# transitions.

Consider the virtual `Activity.OnCreate()` method:

	partial class Activity {
	  static Delegate? cb_onCreate_Landroid_os_Bundle_;
	  static Delegate GetOnCreate_Landroid_os_Bundle_Handler ()
	  {
	    if (cb_onCreate_Landroid_os_Bundle_ == null)
	      cb_onCreate_Landroid_os_Bundle_ = JNINativeWrapper.CreateDelegate ((_JniMarshal_PPL_V) n_OnCreate_Landroid_os_Bundle_);
	    return cb_onCreate_Landroid_os_Bundle_;
	  }
	
	  static void n_OnCreate_Landroid_os_Bundle_ (IntPtr jnienv, IntPtr native__this, IntPtr native_savedInstanceState)
	  {
	    var __this = global::Java.Lang.Object.GetObject<Android.App.Activity> (jnienv, native__this, JniHandleOwnership.DoNotTransfer)!;
	    var savedInstanceState = global::Java.Lang.Object.GetObject<Android.OS.Bundle> (native_savedInstanceState, JniHandleOwnership.DoNotTransfer);
	    __this.OnCreate (savedInstanceState);
	  }
	
	  // Metadata.xml XPath method reference: path="/api/package[@name='android.app']/class[@name='Activity']/method[@name='onCreate' and count(parameter)=1 and parameter[1][@type='android.os.Bundle']]"
	  [Register ("onCreate", "(Landroid/os/Bundle;)V", "GetOnCreate_Landroid_os_Bundle_Handler")]
	  protected virtual unsafe void OnCreate (Android.OS.Bundle? savedInstanceState) => …
	}

`Activity.n_OnCreate_Landroid_os_Bundle_()` is the JNI marshal method,
responsible for marshaling parameters from JNI values into C# types,
forwarding the method invocation to `Activity.OnCreate()`, and (if
necessary) marshaling the return value back to JNI.

`Activity.GetOnCreate_Landroid_os_Bundle_Handler()` is part of the
type registration infrastructure, providing a `Delegate` instance to
`RegisterNativeMembers .RegisterNativeMembers()`, which is eventually
passed to `JNIEnv::RegisterNatives()`.

While this works, it's not incredibly performant: unless using one of
the optimized delegate types (32cff438 et. al),
System.Reflection.Emit is used to create a wrapper around the marshal
method, which is something we've wanted to avoid doing for years.

Thus, the idea: since we're *already* bundling a native toolchain and
using LLVM-IR to produce `libxamarin-app.so` (b21cbf94, 5271f3e1),
what if we emitted [Java Native Method Names][2] and *skipped* all
the done as part of `Runtime.register()` and
`JNIEnv.RegisterJniNatives()`?

Given:

	class MyActivity : Activity {
	    protected override void OnCreate(Bundle? state) => …
	}

During the build, `libxamarin-app.so` would contain the function:

	JNIEXPORT void JNICALL
	Java_crc…_MyActivity_n_1onCreate (JNIEnv *env, jobject self, jobject state);

During App runtime, the `Runtime.register()` invocation present in
[Java Callable Wrappers][3] would either be omitted or would be a
no-op, and Android/JNI would instead resolve `MyActivity.n_onCreate()`
as `Java_crc…_MyActivity_n_1onCreate()`.

Many of the specifics are still being investigated, and this feature
will be spread across various areas.

We call this effort "LLVM Marshal Methods".

First, prepare the way.  Update `Xamarin.Android.Build.Tasks.dll`
and `src/monodroid` to introduce support for generating JNI marshal
methods into `libxamarin-app.so`.  Most of the added code is
*disabled and hidden* behind `#if ENABLE_MARSHAL_METHODS`.

~~ TODO ~~

Other pieces, in no particular order:

  * Update [`Java.Interop.Tools.JavaCallableWrappers`][4] so that
    static constructors aren't needed when LLVM Marshal Methods
    are used.

  * Update [`generator`][5] so that *two* sets of marshal methods are
    emitted: the current set e.g.
    `Activity.n_OnCreate_Landroid_os_Bundle_()`, and an "overload"
    set which has [`UnmanagedCallersOnlyAttribute`][6].
    LLVM Marshal Methods will be able to directly call these
    "unmanaged marshal methods" without the overhead of
    `mono_runtime_invoke()`; see also f48b97cb.

  * Finish the LLVM code generator so that LLVM Marshal Methods are
    emitted into `libxamarin-app.so`.

  * Update the linker so that much of the earlier marshal method
    infrastructure is removed in Release apps.  When
    LLVM Marshal Methods are used, there is no need for
    `Activity.cb_onCreate_Landroid_os_Bundle_`,
    `Actvitiy.GetOnCreate_Landroid_os_Bundle_Handler()`, or the
    `Activity.n_OnCreate_Landroid_os_Bundle_()` without
    `[UnmanagedCallersOnly]`.

Meanwhile, we cannot remove the existing infrastructure, as the
current System.Reflection.Emit-oriented code is needed for faster app
builds, a desirable feature of Debug configuration builds.

LLVM Marshal Methods will be a Release configuration-only feature.

[0]: https://docs.oracle.com/javase/8/docs/technotes/guides/jni/spec/design.html#native_method_arguments
[1]: https://docs.oracle.com/javase/8/docs/technotes/guides/jni/spec/functions.html#RegisterNatives
[2]: https://docs.oracle.com/javase/8/docs/technotes/guides/jni/spec/design.html#resolving_native_method_names
[3]: https://github.com/xamarin/xamarin-android/wiki/Blueprint#java-type-registration
[4]: https://github.com/xamarin/java.interop/tree/main/src/Java.Interop.Tools.JavaCallableWrappers
[5]: https://github.com/xamarin/xamarin-android/wiki/Blueprint#generator
[6]: https://docs.microsoft.com/en-us/dotnet/api/system.runtime.interopservices.unmanagedcallersonlyattribute?view=net-6.0