No description, website, or topics provided.
C# C Other
Clone or download
jonpryor [java-interop] Require dynamic JVM loading (#338)
There is a (*yugely* important) "philosophical" problem with using
`Java.Runtime.Environment.dll.config` to specify the location of the
`jvm.dll` to load, as originated in commit caca88d:

It cannot possibly work *reliably* on any machine *other than* the
build machine.

This is the case because the
`/configuration/dllmap[@dll='jvm.dll']/@target` value comes from the
value of `$(JI_JVM_PATH)`, which is a `make` variable generated during
`make prepare` (see also 4bd9297).  As such, the only way this can
work on an end-users machine is if the end user has the same JDK
version installed as the machine which generated
`Java.Runtime.Environment.dll.config`, which is *highly* unlikely.
(Not everyone on the Xamarin.Android team has the same JDK version!)

The only sane course of action, then, is to *not* use
`Java.Runtime.Environment.dll.config` to specify the location of the
`jvm.dll` to load.  Commit 34129b6 started implementing this; let's
finish the effort, and require it everywhere.

*A* reason that commit 34129b6 kept
`Java.Runtime.Environment.dll.config` use was so that unit tests would
continue to execute, as the `TestJVM` type uses
`Java.Runtime.Environment.dll`, which uses
`Java.Runtime.Environment.dll.config`.

To support this scenario, update the `<JdkInfo/>` task so that the
generated `JdkInfo.mk` file *`export`*s the `$(JI_JVM_PATH)` variable.
This allows unit tests executed from e.g. `make run-all-tests` to
inherit the `$JI_JVM_PATH` environment variable, allowing them to
"implicitly explicitly" specify the location of the JVM to use.

With unit test support in place, we can rip out all use of `jvm.dll`,
requiring that we instead use e.g.
`java-interop!java_interop_jvm_load()` to load the JVM.

Additionally, reduce the "public API surface" within
`java-interop-jvm.h` by moving the declaration of `DylibJVM` into
`java-interop-jvm.c`.  The `DylibJVM` type doesn't need to be public,
and making it private will allow us to more easily change it.
Latest commit d1cce19 Jun 28, 2018
Permalink
Failed to load latest commit information.
Documentation Review, update, and flush docs. Apr 26, 2016
build-tools [xamarin-android-docimporter-ng] Final API-28 API update (#332) Jun 11, 2018
external Bump to cecil/master/b6c50e3d34 (#296) Apr 13, 2018
lib Bump to mono.linq.expressions/e3372fbb (#70) Sep 1, 2016
samples/Hello [*-Tests] Move output into bin\Test$(Configuration) Mar 29, 2016
src [java-interop] Require dynamic JVM loading (#338) Jun 27, 2018
tests [java-interop] Require dynamic JVM loading (#338) Jun 27, 2018
tools [java-interop] Require dynamic JVM loading (#338) Jun 27, 2018
.gitattributes [build] Use .csproj files, not .mdproj files (#277) Mar 28, 2018
.gitignore [build] setup PrepareWindows.targets (#285) Apr 7, 2018
.gitmodules Bump to cecil/master/b6c50e3d34 (#296) Apr 13, 2018
Before.Java.Interop.sln.targets [tests] use wildcards for simpler generator integration tests (#290) Apr 11, 2018
Configuration.Override.props.in [Java.Runtime.Environment] Use libjli.dylib on macOS, not libjvm.dylib ( Oct 25, 2017
Configuration.props [build] Allow overriding Cecil location (#328) May 30, 2018
Java.Interop.sln [build] Use .csproj files, not .mdproj files (#277) Mar 28, 2018
LICENSE Add MIT/X11 License. Apr 26, 2016
Makefile [java-interop] Require dynamic JVM loading (#338) Jun 27, 2018
README.md [docs] fix typo (#337) Jun 20, 2018
gendarme-ignore.txt [java-interop] Require dynamic JVM loading (#338) Jun 27, 2018
product.snk [Java.Interop, Java.Interop.Dynamic, Java.Interop.Export] Sign assemb… May 10, 2016

README.md

Java.Interop

Gitter

Java.Interop is a binding of the Java Native Interface for use from managed languages such as C#, and an associated set of code generators to allow Java code to invoke managed code. It is also a brain-delusional Second System Syndrome rebuild of the monodroid/Xamarin.Android core, intended to fix some of the shortcomings and design mistakes I've made over the years.

In particular, it attempts to fix the following issues:

  • Split out the core invocation logic so that the containing assembly is in the xbuild-frameworks\MonoAndroid\v1.0 directory, allowing low-level JNI use without taking an API-level constraint.
  • Make the assembly a PCL lib.
  • Support use of the lib on "desktop" Java VMs. This would allow more testing without an Android device, could allow using Xamarin.Android Views to be shown in the GUI designer, etc.
  • Improve type safety.
  • Improve consistency.

In particular are the last two points: Xamarin.Android currently uses IntPtrs everywhere, and it's not at all obvious what they are (method IDs vs. local refs vs. global refs vs. ...). This culminates in JNIEnv.FindClass(), which returns a global reference while most other methods return a local ref.

The JNIEnv API is also huge, unwieldy, and terrible.

Build Requirements

Latest Mono from http://www.mono-project.com/

Build Configuration

The Java.Interop build can be configured by overriding make(1) variables on the command line or by specifying MSBuild properties to control behavior.

make(1) variables

The following make(1) variables may be specified:

  • $(CONFIGURATION): The product configuration to build, and corresponds to the $(Configuration) MSBuild property when running $(MSBUILD). Valid values are Debug and Release. Default value is Debug.

  • $(RUNTIME): The managed runtime to use to execute utilities, tests. Default value is mono64 if present in $PATH, otherwise mono.

  • $(TESTS): Which unit tests to execute. Useful in conjunction with the make run-tests target:

      make run-tests TESTS=bin/Debug/Java.Interop.Dynamic-Tests.dll
    
  • $(V): If set to a non-empty string, adds /v:diag to $(MSBUILD_FLAGS) invocations.

  • $(MSBUILD): The MSBuild build tool to execute for builds. Default value is xbuild.

MSBuild Properties

MSbuild properties may be placed into the file Configuration.Override.props, which can be copied from Configuration.Override.props.in. The Configuration.Override.props file is <Import/>ed by Configuration.props; there is no need to <Import/> it within other project files.

Overridable MSBuild properties include:

  • $(CecilSourceDirectory): Directory for the cecil sources. Defaults to external/cecil.
  • $(JdkJvmPath): Full path name to the JVM native library to link java-interop against. By default this is probed for from numerous locations within build-tools/scripts/jdk.mk.
  • $(JavaCPath): Path to the javac command-line tool, by default set to javac.
  • $(JarPath): Path to the jar command-line tool, by default set to jar.
    • It may be desirable to override these on Windows, depending on your PATH.
  • $(UtilityOutputFullPath): Directory to place various utilities such as class-parse, generator, and logcat-parse. This value should be a full path. By default this is $(MSBuildThisFileDirectory)bin/$(Configuration).

Type Safety

The start of the reboot was to use strongly typed SafeHandle subclasses everywhere instead of IntPtr. This allows a local reference to be type-checked and distinct from a global ref, complete with compiler type checking.

Since we now have actual types in more places, we can move the current JNIEnv methods into more semantically meaningful types.

Unfortunately, various tests demonstrated that while SafeHandles provided increased type safety, they did so at a large runtime cost:

  1. SafeHandles are reference types, increasing GC heap allocations and pressure.
  2. SafeHandles are thread-safe in order to prevent race conditions and handle recycling attacks.

Compared to a Xamarin.Android-like "use IntPtrs for everything" binding approach, the overhead is significant: to just invoke JNIEnv::CallObjectMethod(), using SafeHandles for everything causes execution time to take ~1.4x longer than a comparable struct-oriented approach.

Make the test more realistic -- compared to current Xamarin.Android and current Java.Interop -- so that JniEnvironment.Members.CallObjectMethod() also calls JniEnvironment.Errors.ExceptionOccurred(), which also returns a JNI local reference -- and runtime execution time jumped to ~3.6x:

# SafeHandle timing: 00:00:09.9393493
#	Average Invocation: 0.00099393493ms
# JniObjectReference timing: 00:00:02.7254572
#	Average Invocation: 0.00027254572ms

(See the tests/invocation-overhead directory for the invocation comparison sourcecode.)

This is not acceptable. Performance is a concern with Xamarin.Android; we can't be making it worse.

Meanwhile, I really dislike using IntPtrs everywhere, as it doesn't let you know what the value actually represents.

To solve this issue, avoid SafeHandle types in the public API.

Downside: this means we can't have the GC collect our garbage JNI references.

Upside: the Java.Interop effort will actually be usable.

Instead of using SafeHandle types, we introduce a JniObjectReference struct type. This represents a JNI Local, Global, or WeakGlobal object reference. The JniObjectReference struct also contains the reference type as JniObjectReferenceType. jmethodID and jfieldID become "normal" class types, permitting type safety, but lose their SafeHandle status, which was never really necessary because they don't require cleanup anyway. Furthermore, these values should be cached -- see JniPeerMembers -- so making them GC objects shouldn't be a long-term problem.

By doing so, we allow Java.Interop to have two separate implementations, controlled by build-time #defines:

  • FEATURE_HANDLES_ARE_SAFE_HANDLES: Causes JniObjectReference to contain a SafeHandle wrapping the underlying JNI handle.
  • FEATURE_HANDLES_ARE_INTPTRS: Causes JniObjectReference to contain an IntPtr for the underlying JNI handle.

The rationale for this is twofold:

  1. It allows swapping out "safer" SafeHandle and "less safe" IntPtr implementations, permitting easier performance comparisons.
  2. It allows migrating the existing code, as some of the existing tests may assume that JNI handles are garbage collected, which won't be the case when FEATURE_HANDLES_ARE_INTPTRS is set.

Naming Conventions

Types with a Java prefix are "high-level" types which participate in cross-VM object-reference semantics, e.g. you could add a JavaObject subclass to a Java-side collection, perform a GC, and the instance will survive the GC.

Types with a Jni prefix are "low-level" types and do not participate in object-reference semantics.

Android Tests

src/Android.Interop is vestigial; it was for testing before a subset of Java.Interop was integrated with Xamarin.Android 6.1 ("cycle 7"). It should arguably be deleted.

The top-level make run-android target will run the Java.Interop unit tests on Android via the Android.Interop-Tests project.

The Android.Interop-Tests project currently contains all tests, including the time intensive "PerformanceTests".

To run a specific test fixture, set the FIXTURE variable:

make run-android FIXTURE=Java.Interop.PerformanceTests.TimingTests

Notes

JDK and Global References

The JDK VM supports an effectively unlimited number of global references. While Dalvik bails out after creating ~64k GREFs, consider the following on the JDK:

var t = new JniType ("java/lang/Object");
var c = t.GetConstructor ("()V");
var o = t.NewInstance (c);
int count = 0;
while (true) {
    Console.WriteLine ("count: {0}", count++);
    o.NewGlobalRef ();
}

I halted the above loop after reaching 25686556 instances.

count: 25686556
^C

I'm not sure when the JDK would stop handing out references, but it's probably bound to process heap limits (e.g. depends on 32-bit vs. 64-bit process).

Building on Windows

Building Java.Interop on Windows requires .NET and the msbuild command be available within the Command-Line environment. (The Developer Command Prompt that Visual Studio installs is sufficient.)

MSBuild version 15 or later is required.

  1. Install the build requirements.

  2. Clone the java.interop repo:

    git clone https://github.com/xamarin/java.interop.git
    
  3. Navigate to the java.interop directory

  4. Prepare the project:

    msbuild Java.Interop.sln /t:Prepare
    

    This will ensure that the build dependencies are installed, perform git submodule update, download NuGet dependencies, and other "preparatory" and pre-build tasks that need to be performed.

  5. Build the project (or open and build with an IDE):

    msbuild Java.Interop.sln