proposal: expose Java API to gomobile bind programs

[eliasnaur]

Abstract

Today, gomobile bind can take a set og Go packages and expose their public API to Java or ObjC apps. The proposal is to support the reverse, exposing Java API to the bound Go packages.

Motivation

Even though mobile apps can access Go code already, there are still large parts of a typical app that is impossible or awkward to implement in Go. The most notable case is UI code, which interacts most with the platform APIs.
Platform APIs can be accessed from Go in an indirect way already. Creating a Java (or ObjC) class wrapping the desired API and passing it to Go does work. However, this is not nearly as convenient as writing the code directly in Java.
To improve support for writing platform specific code in Go, direct access to the platform is needed.

Proposed features

Importing Java classes and interfaces from Go

The Go wrappers for all Java API are generated each time gomobile bind is called. To access a Java package, use import statements on the form:

    import "Java/some/pkg"

To access the static methods or constants on a Java class or interface, use

    import "Java/some/pkg/SomeClass"

or

    import "Java/some/pkg/SomeClass/InnerClass"

for an inner class.

Static methods and constants

After importing, the resulting packages SomeClass and InnerClass will contain the static methods and static final constants from their Java classes. For example

import "Java/java/lang/Float"

will expose (among others) the constant Float.MIN_VALUE and the function Float.ParseFloat.

Java classes and interfaces

The package "Java/some/pkg" contains Go interfaces wrapping every referenced Java type in some.pkg. The wrapper types are used to represent their wrapped Java types across the language barrier and to call methods on wrapped instances. For example, with the following Go function is now possible:

import "Java/java/lang"

func FloatDoubleValue(f lang.Float) float64 {
    return f.DoubleValue()
}

Creating new Java instances

To create a new instance of a Java class, use the New function defined in the class package. For example:

import (
    "Java/java/lang/Object"
    "Java/java/lang"
)

func NewObject() lang.Object {
    return Object.New()
}

Errors and exceptions

Exceptions are normally translated to explicit Go errors, but since we don't control the platform API, we don't know which Java methods can result in an exception worth catching. Instead, a simple heuristic is used: If a Java method is declared to throw one or more exceptions, its Go function or method will return an error. If no exception is declared, any exception thrown will be converted to a panic with the exception as argument.

In addition, any Java class which inherits from java.lang.Throwable will satisfy the error interface. Its Error method will delegate to the toString() method.

Extending or implementing Java types from Go

Gomobile already exposes exported Go structs to Java; this proposal adds support for constructing Go structs directly from Java. In addition, Go structs will be able to extend Java classes and implement Java interfaces.

To declare a Go struct that extends or implements Java types, use the form:

import "Java/some/pkg/Class"
import "Java/some/pkg/Inner"
import "Java/another/pkg2/Interface"

type S struct {
    pkg.Class // extends Class
    pkg2.Interface // implements Interface
    Class.Inner // implements (or extends) inner interface (or class)
}

Java constructors

To allow Java to create instances of a Go struct, S, add one or more constructor on the form:

func NewS(...) *S {
    ...
}

For each such Go constructor a Java constructor will be added taking the same arguments. The Java constructor calls its super constructor with its arguments before calling calling NewS. For example:

package gopkg

import (
    "Java/java/lang"
)

type GoObject struct {
    lang.Object
}

func NewGoObject() *GoObject {
    return &GoObject{}
)

will allow Java to construct instances of GoObject:

import go.gopkg.GoObject;

...

    GoObject o = new GoObject();

Overriding Java methods

To implement or override a method from a super class or interface, declare a Go method with the same name and its first letter capitalized. For example, to override the toString method in GoObject:

func (o *GoObject) ToString() string {
    ...
}

Exposing this

Whenever an foreign object is passed across the language barrier, a proxy is created to represent it. In the example above, there is a GoObject Java instance created in Java, and it contains a reference to its counterpart GoObject Go instance in Go. That means that when a Go method is called from Java, its method receiver contains the Go instance, while the Java instance is only accessible to Java.
To access the Java instance (for passing back to other Java APIs), any Go method can declare a this argument with one of the Java types the enclosing class extends or implements. For example, to access the this from the ToString method, use:

func (o *GoObject) ToString(t lang.Object) string {
    ...
}

The t variable will behave just as if it were a pure Java Object, and if passed to Java, it will have the same identity as the Java reference.

Calling super

In Go, delegation is achieved through delegation, but in Java, the keyword super is needed to access overridden methods. To call a super method from Go, use the Super() method on the this variable:

func (o *GoObject) ToString(t lang.Object) string {
    return t.Super().ToString()
}

Overloaded methods and constructors

Java supports overloading; Go doesn't. To access or override overloaded methods and cosntructors from Go, a mangling scheme is used:

• For any overloaded method where the number of arguments uniquely identifies the method, the argument count is appended to its name, except if the methods takes no arguments. For example, the Java methods

void m();
void m(int i);

are called M and M2, respectively, in Go.

• If multiple methods have the same name and the same number of arguments, their names have an underscore and the JNI mangled argument descriptor appended. For example, the Java methods:

void m(int i);
void m(String s)

are called M_I and M_Ljava_lang_String_2 in Go.

The JNI name mangling scheme is ugly. In particular, Java constructors are only distinguished by their arguments and are therefore often mangled. Suggestions for improved schemes are most welcome!

[gopherbot]

CL https://golang.org/cl/27751 mentions this issue.

[kardianos]

I have no opinion on this directly. But if this is considered, would it be possible to use some sort of prefix char to denote this package is provided by the build tool. "*Java/java/lang"

Rational: currently package tools need to understand special packages like "C". While it wouldn't be the end of the world to add other common special cases (starts with "Java/" or "objc/") it might be worth adding a convention for them.

If this was generalized:

import (
    "*C"
    "*Java/java/Object"
    "*appengine"
)

[eliasnaur]

@kardianos I would prefer the current naming scheme similar to Cgo's import "C". Adding the * special case for just gomobile seems like overkill. Besides, I wouldn't be surprised if the Java/* packages were someday pre-generated and thus accessible to tools as any other Go package.

(also, the "appengine" import is a misleading example, since it is being replaced by the go gettable "google.golang.org/appengine..." family of packages)

[kardianos]

@eliasnaur That's fine. It is just another thing tools that deal with package tools will need to special case. If they are pre-generated, they would have full normal go import paths discoverable on disk. If you noticed the final example, you could also use for current appengine and C imports as well.

[mrkaspa]

I really like this idea not only for mobile dev, the proposal should also specify how to vendor java dependencies

[nodirt]

The proposal doesn't address one of the toughest problems that also arise in go->c++ interop, one of the reasons why libs are rewritten in pure Go: when a goroutine calls a blocking Java API, should entire OS thread be occupied by Java Run-time and just sit idle while it could be used by other goroutines?

[eliasnaur]

@nodirt If it is possible to rewrite in pure Go, go ahead! This proposal is for accessing (Android) API that is otherwise not available to Go. Can you think of an Android API that takes up a significant amount of threads? Even if you can, I don't see how it is possible to avoid the problem from Go. One thread per blocking call is the execution model of Java.

[nodirt]

I'm not suggesting to rewrite anything.

It is not about API that takes up more than one thread, but about blocking calls. For example, imagine a goroutine calls FileOutputStream.write to write a file locally (it may be a bad example, I'm not familiar with Android API. You can probably come up with a better example); this is a blocking call. In a typical go app a goroutine waiting for a blocking call doesn't hold a thread, but let's other goroutines use it. Java doesn't do that, obviously; an entire thread will be just waiting for the blocking call to return. If GOMAXPROCS=4 and 4 goroutines made blocking calls into Java API, all threads may be waiting for the blocking calls to return and program may freeze.

Maybe you have a solution to this problem, but it is not obvious from this proposal

[eliasnaur]

@nodirt As far as I know, Go programs never freezes or deadlocks because of blocking C call. Gomobile is implemented through Cgo, and Cgo assumes every C call might block, so it creates new threads as necessary for goroutines to run on.

There is even an issue about optimizing the non-blocking case: #16051

[teknico]

However, this is not nearly as convenient as writing the code directly in Java.

You meant "...directly in Go", right? :-)

[eliasnaur]

@teknico Actually I didn't :) The point I'm trying to convey is that this proposal doesn't enable access to platform APIs (they're already indirectly accessible) as much as improve the convenience of access.

[dskinner]

For overloaded methods, I disagree with having two different methods for determining the name. The first method of appending number of arguments is simply deficient for handling a number of cases and should be discarded.

I think the full bytecode name of the object appended is overkill but can't confirm otherwise. I think the likelihood of an overloaded method that has positional arguments with the same class name but different namespaces is simply nonexistent.

Intent has a long list of overloads for putExtra. The first argument is always a string with the second argument differing. Using only the differences for distinguishing overloaded methods, one could derive more friendly names, putExtraString, putExtraInteger.

Still, that's short-sighted and I think it'd be worthwhile to list out all the overloaded methods from android.jar to see what kind of cases can be expected instead of assuming worse-case with use of bytecode names.

[dskinner]

I printed out a list of overloaded methods under android.* and I think MotionEvent.obtain would make a good case for scrutiny.

public static android.view.MotionEvent obtain(long, long, int, int, android.view.MotionEvent$PointerProperties[], android.view.MotionEvent$PointerCoords[], int, int, float, float, int, int, int, int)
public static android.view.MotionEvent obtain(long, long, int, int, int[], android.view.MotionEvent$PointerCoords[], int, float, float, int, int, int, int)
public static android.view.MotionEvent obtain(long, long, int, float, float, float, float, int, float, float, int, int)
public static android.view.MotionEvent obtain(long, long, int, int, float, float, float, float, int, float, float, int, int)
public static android.view.MotionEvent obtain(long, long, int, float, float, int)
public static android.view.MotionEvent obtain(android.view.MotionEvent)

Two of the methods contain the same number of arguments (13) making the the first proposed rule for naming ineffective and leaving an unusually long name for the first method listed.

[eliasnaur]

I believe multiple methods are necessary for readability. Only the last rule has to cover everything, and then it might as well be the JNI rules. For JNI methods in C, there in a sense already two methods: (1) Use the method name if it is not overloaded (2) If it is overloaded, mangle the argument types to disambiguate. From that view, I'm merely introducing a second, much more readable, fallback before the final catch-all rule.