proposal: expose Java API to gomobile bind programs #16876

Closed
eliasnaur opened this Issue Aug 25, 2016 · 24 comments

10 participants

@eliasnaur
Contributor
eliasnaur commented Aug 25, 2016 edited

Abstract

Today, gomobile bind can take a set og Go packages and expose their public API to Java or ObjC apps. The proposal is to support the reverse, exposing Java API to the bound Go packages.

Motivation

Even though mobile apps can access Go code already, there are still large parts of a typical app that is impossible or awkward to implement in Go. The most notable case is UI code, which interacts most with the platform APIs.
Platform APIs can be accessed from Go in an indirect way already. Creating a Java (or ObjC) class wrapping the desired API and passing it to Go does work. However, this is not nearly as convenient as writing the code directly in Java.
To improve support for writing platform specific code in Go, direct access to the platform is needed.

Proposed features

Importing Java classes and interfaces from Go

The Go wrappers for all Java API are generated each time gomobile bind is called. To access a Java package, use import statements on the form:

    import "Java/some/pkg"

To access the static methods or constants on a Java class or interface, use

    import "Java/some/pkg/SomeClass"

or

    import "Java/some/pkg/SomeClass/InnerClass"

for an inner class.

Static methods and constants

After importing, the resulting packages SomeClass and InnerClass will contain the static methods and static final constants from their Java classes. For example

import "Java/java/lang/Float"

will expose (among others) the constant Float.MIN_VALUE and the function Float.ParseFloat.

Java classes and interfaces

The package "Java/some/pkg" contains Go interfaces wrapping every referenced Java type in some.pkg. The wrapper types are used to represent their wrapped Java types across the language barrier and to call methods on wrapped instances. For example, with the following Go function is now possible:

import "Java/java/lang"

func FloatDoubleValue(f lang.Float) float64 {
    return f.DoubleValue()
}

Creating new Java instances

To create a new instance of a Java class, use the New function defined in the class package. For example:

import (
    "Java/java/lang/Object"
    "Java/java/lang"
)

func NewObject() lang.Object {
    return Object.New()
}

Errors and exceptions

Exceptions are normally translated to explicit Go errors, but since we don't control the platform API, we don't know which Java methods can result in an exception worth catching. Instead, a simple heuristic is used: If a Java method is declared to throw one or more exceptions, its Go function or method will return an error. If no exception is declared, any exception thrown will be converted to a panic with the exception as argument.

In addition, any Java class which inherits from java.lang.Throwable will satisfy the error interface. Its Error method will delegate to the toString() method.

Extending or implementing Java types from Go

Gomobile already exposes exported Go structs to Java; this proposal adds support for constructing Go structs directly from Java. In addition, Go structs will be able to extend Java classes and implement Java interfaces.

To declare a Go struct that extends or implements Java types, use the form:

import "Java/some/pkg/Class"
import "Java/some/pkg/Inner"
import "Java/another/pkg2/Interface"

type S struct {
    pkg.Class // extends Class
    pkg2.Interface // implements Interface
    Class.Inner // implements (or extends) inner interface (or class)
}

Java constructors

To allow Java to create instances of a Go struct, S, add one or more constructor on the form:

func NewS(...) *S {
    ...
}

For each such Go constructor a Java constructor will be added taking the same arguments. The Java constructor calls its super constructor with its arguments before calling calling NewS. For example:

package gopkg

import (
    "Java/java/lang"
)

type GoObject struct {
    lang.Object
}

func NewGoObject() *GoObject {
    return &GoObject{}
)

will allow Java to construct instances of GoObject:

import go.gopkg.GoObject;

...

    GoObject o = new GoObject();

Overriding Java methods

To implement or override a method from a super class or interface, declare a Go method with the same name and its first letter capitalized. For example, to override the toString method in GoObject:

func (o *GoObject) ToString() string {
    ...
}

Exposing this

Whenever an foreign object is passed across the language barrier, a proxy is created to represent it. In the example above, there is a GoObject Java instance created in Java, and it contains a reference to its counterpart GoObject Go instance in Go. That means that when a Go method is called from Java, its method receiver contains the Go instance, while the Java instance is only accessible to Java.
To access the Java instance (for passing back to other Java APIs), any Go method can declare a this argument with one of the Java types the enclosing class extends or implements. For example, to access the this from the ToString method, use:

func (o *GoObject) ToString(t lang.Object) string {
    ...
}

The t variable will behave just as if it were a pure Java Object, and if passed to Java, it will have the same identity as the Java reference.

Calling super

In Go, delegation is achieved through delegation, but in Java, the keyword super is needed to access overridden methods. To call a super method from Go, use the Super() method on the this variable:

func (o *GoObject) ToString(t lang.Object) string {
    return t.Super().ToString()
}

Overloaded methods and constructors

Java supports overloading; Go doesn't. To access or override overloaded methods and cosntructors from Go, a mangling scheme is used:

  • For any overloaded method where the number of arguments uniquely identifies the method, the argument count is appended to its name, except if the methods takes no arguments. For example, the Java methods
void m();
void m(int i);

are called M and M2, respectively, in Go.

  • If multiple methods have the same name and the same number of arguments, their names have an underscore and the JNI mangled argument descriptor appended. For example, the Java methods:
void m(int i);
void m(String s)

are called M_I and M_Ljava_lang_String_2 in Go.

The JNI name mangling scheme is ugly. In particular, Java constructors are only distinguished by their arguments and are therefore often mangled. Suggestions for improved schemes are most welcome!

@gopherbot

CL https://golang.org/cl/27751 mentions this issue.

@quentinmit quentinmit added this to the Proposal milestone Aug 26, 2016
@kardianos
Contributor
kardianos commented Sep 1, 2016 edited

I have no opinion on this directly. But if this is considered, would it be possible to use some sort of prefix char to denote this package is provided by the build tool. "*Java/java/lang"

Rational: currently package tools need to understand special packages like "C". While it wouldn't be the end of the world to add other common special cases (starts with "Java/" or "objc/") it might be worth adding a convention for them.

If this was generalized:

import (
    "*C"
    "*Java/java/Object"
    "*appengine"
)
@eliasnaur
Contributor
eliasnaur commented Sep 1, 2016 edited

@kardianos I would prefer the current naming scheme similar to Cgo's import "C". Adding the * special case for just gomobile seems like overkill. Besides, I wouldn't be surprised if the Java/* packages were someday pre-generated and thus accessible to tools as any other Go package.

(also, the "appengine" import is a misleading example, since it is being replaced by the go gettable "google.golang.org/appengine..." family of packages)

@kardianos
Contributor

@eliasnaur That's fine. It is just another thing tools that deal with package tools will need to special case. If they are pre-generated, they would have full normal go import paths discoverable on disk. If you noticed the final example, you could also use for current appengine and C imports as well.

@mrkaspa
mrkaspa commented Sep 1, 2016

I really like this idea not only for mobile dev, the proposal should also specify how to vendor java dependencies

@nodirt
Member
nodirt commented Sep 1, 2016

The proposal doesn't address one of the toughest problems that also arise in go->c++ interop, one of the reasons why libs are rewritten in pure Go: when a goroutine calls a blocking Java API, should entire OS thread be occupied by Java Run-time and just sit idle while it could be used by other goroutines?

@eliasnaur
Contributor

@nodirt If it is possible to rewrite in pure Go, go ahead! This proposal is for accessing (Android) API that is otherwise not available to Go. Can you think of an Android API that takes up a significant amount of threads? Even if you can, I don't see how it is possible to avoid the problem from Go. One thread per blocking call is the execution model of Java.

@nodirt
Member
nodirt commented Sep 2, 2016 edited

I'm not suggesting to rewrite anything.

It is not about API that takes up more than one thread, but about blocking calls. For example, imagine a goroutine calls FileOutputStream.write to write a file locally (it may be a bad example, I'm not familiar with Android API. You can probably come up with a better example); this is a blocking call. In a typical go app a goroutine waiting for a blocking call doesn't hold a thread, but let's other goroutines use it. Java doesn't do that, obviously; an entire thread will be just waiting for the blocking call to return. If GOMAXPROCS=4 and 4 goroutines made blocking calls into Java API, all threads may be waiting for the blocking calls to return and program may freeze.

Maybe you have a solution to this problem, but it is not obvious from this proposal

@eliasnaur
Contributor
eliasnaur commented Sep 2, 2016 edited

@nodirt As far as I know, Go programs never freezes or deadlocks because of blocking C call. Gomobile is implemented through Cgo, and Cgo assumes every C call might block, so it creates new threads as necessary for goroutines to run on.

There is even an issue about optimizing the non-blocking case: #16051

@teknico
teknico commented Sep 2, 2016

However, this is not nearly as convenient as writing the code directly in Java.

You meant "...directly in Go", right? :-)

@eliasnaur
Contributor

@teknico Actually I didn't :) The point I'm trying to convey is that this proposal doesn't enable access to platform APIs (they're already indirectly accessible) as much as improve the convenience of access.

@dskinner
Member
dskinner commented Sep 6, 2016

For overloaded methods, I disagree with having two different methods for determining the name. The first method of appending number of arguments is simply deficient for handling a number of cases and should be discarded.

I think the full bytecode name of the object appended is overkill but can't confirm otherwise. I think the likelihood of an overloaded method that has positional arguments with the same class name but different namespaces is simply nonexistent.

Intent has a long list of overloads for putExtra. The first argument is always a string with the second argument differing. Using only the differences for distinguishing overloaded methods, one could derive more friendly names, putExtraString, putExtraInteger.

Still, that's short-sighted and I think it'd be worthwhile to list out all the overloaded methods from android.jar to see what kind of cases can be expected instead of assuming worse-case with use of bytecode names.

@dskinner
Member
dskinner commented Sep 6, 2016

I printed out a list of overloaded methods under android.* and I think MotionEvent.obtain would make a good case for scrutiny.

public static android.view.MotionEvent obtain(long, long, int, int, android.view.MotionEvent$PointerProperties[], android.view.MotionEvent$PointerCoords[], int, int, float, float, int, int, int, int)
public static android.view.MotionEvent obtain(long, long, int, int, int[], android.view.MotionEvent$PointerCoords[], int, float, float, int, int, int, int)
public static android.view.MotionEvent obtain(long, long, int, float, float, float, float, int, float, float, int, int)
public static android.view.MotionEvent obtain(long, long, int, int, float, float, float, float, int, float, float, int, int)
public static android.view.MotionEvent obtain(long, long, int, float, float, int)
public static android.view.MotionEvent obtain(android.view.MotionEvent)

Two of the methods contain the same number of arguments (13) making the the first proposed rule for naming ineffective and leaving an unusually long name for the first method listed.

@eliasnaur
Contributor

I believe multiple methods are necessary for readability. Only the last rule has to cover everything, and then it might as well be the JNI rules. For JNI methods in C, there in a sense already two methods: (1) Use the method name if it is not overloaded (2) If it is overloaded, mangle the argument types to disambiguate. From that view, I'm merely introducing a second, much more readable, fallback before the final catch-all rule.

@gopherbot

CL https://golang.org/cl/28596 mentions this issue.

@gopherbot

CL https://golang.org/cl/28597 mentions this issue.

@gopherbot

CL https://golang.org/cl/28595 mentions this issue.

@slimsag
slimsag commented Sep 7, 2016

One thing I don't see this proposal discussing is why this needs to be a language extension / cannot be done as a standard Go package (cgo bindings to the JNI).

@eliasnaur
Contributor

It's all implemented in golang.org/x/mobile. Nothing changes in Go itself.

@eliasnaur
Contributor

I've updated the implementation and proposal to replace the crude Go import scanning with a more sophisticated method of determining what Java types and identifiers are used.
It is no longer necessary to import java classes or interfaces to have their wrappers generated. Also, the amount of wrapper code is greatly reduced, because only (potentially) referenced methods and functions are included.

@gopherbot

CL https://golang.org/cl/29172 mentions this issue.

@dskinner
Member

Only the last rule has to cover everything, and then it might as well be the JNI rules.

sgtm, it does at least afford a basis to build upon.

@gopherbot gopherbot pushed a commit to golang/mobile that referenced this issue Sep 16, 2016
@eliasnaur eliasnaur internal/importers: introduce package to analyze Java classes
The importers package adds functions to traverse the AST of a Go
file or set of packages and extract references to Java packages and
types.

The java package adds a parser that uses the javap command to extract
type information about Java classes and interfaces.

The resulting type information is needed to generate Java API wrappers
in Go.

This is the first part of the implementation of proposal golang/go#16876.

For golang/go#16876

Change-Id: Ic844472a1101354d61401d9e8c120acdee2519df
Reviewed-on: https://go-review.googlesource.com/28595
Reviewed-by: David Crawshaw <crawshaw@golang.org>
e4531be
@gopherbot gopherbot pushed a commit to golang/mobile that referenced this issue Sep 16, 2016
@eliasnaur eliasnaur bind,cmd/gomobile: add a new generator for Java API wrappers
Using the new Java class analyzer API, scan the bound packages
for references to Java classes and interfaces and generate Go
wrappers for them.

This is the second part of the implementation of proposal golang/go#16876.

For golang/go#16876

Change-Id: I59ec0ebdae0081a615dc34d450f344c20c03f871
Reviewed-on: https://go-review.googlesource.com/28596
Reviewed-by: David Crawshaw <crawshaw@golang.org>
bf31dd1
@gopherbot gopherbot pushed a commit to golang/mobile that referenced this issue Sep 22, 2016
@eliasnaur eliasnaur bind,cmd: accept Java API in bound packages
Accept Java API interface types as arguments and return values from
bound Go package functions and methods. Also, allow Go structs
to extend Java classes and implement Java interfaces as well as override
and implement methods.

This is the third and final part of the implementation of the golang/go#16876
proposal.

Fixes golang/go#16876

Change-Id: I6951dd87235553ce09abe5117a39a503466163c0
Reviewed-on: https://go-review.googlesource.com/28597
Reviewed-by: David Crawshaw <crawshaw@golang.org>
bdf873e
@gopherbot gopherbot pushed a commit to golang/mobile that closed this issue Sep 22, 2016
@eliasnaur eliasnaur bind,cmd: accept Java API in bound packages
Accept Java API interface types as arguments and return values from
bound Go package functions and methods. Also, allow Go structs
to extend Java classes and implement Java interfaces as well as override
and implement methods.

This is the third and final part of the implementation of the golang/go#16876
proposal.

Fixes golang/go#16876

Change-Id: I6951dd87235553ce09abe5117a39a503466163c0
Reviewed-on: https://go-review.googlesource.com/28597
Reviewed-by: David Crawshaw <crawshaw@golang.org>
bdf873e
@cable309
cable309 commented Mar 25, 2017 edited

Sorry. but I just want to know how to get a public attribute in a java class ?
for example:
realPackageInfo := PackageInfo.Cast(packageInfo) fmt.Println(realPackageInfo.InstallLocation())
I want to get the packageinfo's installlocation attribute ,and it's public . how can I do this?
I checked all the examples in the gomobile library, but didn't find the answer.

It just cause this error. but when I call a function of the PackageInfo it can compile correctly.

./GO2.go:21: realPackageInfo.InstallLocation undefined (type Java.Android_content_pm_PackageInfo has no field or method InstallLocation)

@eliasnaur
Contributor

I'm sorry, but Java fields are not (yet) exposed by reverse bindings.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment