How active is this project?

[Thrameos]

I am the author of JPype which is another one of the Python to Java bridges. As part of my effort to consolidate the bridge codes to improve functionality and further compatibility between Java using Python modules, I am reaching out to the other Java bridges to see what state they are in and if there is useful distinguishing differences between the packages. Some efforts are largely duplicative such as PyJNIus vs JPype. JPype had many more features but PyJNIus was being developed because JPype lacked support for Android which was important to the PyJNIus authors. In other cases the bridges serve completely different purposes (JPype vs Py4j) and are unlike to ever be consolidated.

As far as I can assess from the javabridge documentation this is a much lower level API that JPype. It seems like it focuses on direct JNI calls where the signature has to be specified. It is possible to directly call methods using JPype though most users use the automatic dispatches rather than requiring specifying calls directly. Is is not clear from the JavaBridge documentation if it handles the edge cases such as caller sensitive methods, joining the garbage collectors, array or buffer transfers, or speed of calls.

Is this an active project? If not are there features in this that are not available in JPype that need to be added to convert the same space? Are there specialty features that warrant keeping it as a separate project (license, features, size, application space, etc)?

[LeeKamentsky]

The project is pretty much in a maintenance mode. The primary users are people who want to use Weka with Python and for the CellProfiler / ImageJ community. As I understand it, JPype works out of process, so possibly that is a distinction. There's no monetary support for the project, although it was a large part of my work, perhaps 5-10 years ago, I maintain it mostly as a courtesy to the community. I'd say the chief positive for Javabridge is its coupling with Numpy and Numpy arrays which is useful in the context of image processing and scientific computation. There is a very low-level API that maps the JNI pretty closely, but it is possible to use some of the reflection-sensitive parts of Javabridge to create Python objects that wrap Java ones and dot notations to run methods. This part of the code is simple to use, but the reflection is far from computationally efficient - a good use case is connecting to a Java method that does heavy computation that amortizes the cost of Javabridge's reflection. There are packages that depend on the specifics of Javabridge. That's probably the reason to keep it around unless you want to find out who uses it and get them to port.

…

On Fri, Nov 20, 2020 at 2:15 PM Karl Nelson ***@***.***> wrote: I am the author of JPype which is another one of the Python to Java bridges. As part of my effort to consolidate the bridge codes to improve functionality and further compatibility between Java using Python modules, I am reaching out to the other Java bridges to see what state they are in and if there is useful distinguishing differences between the packages. Some efforts are largely duplicative such as PyJNIus vs JPype. JPype had many more features but PyJNIus was being developed because JPype lacked support for Android which was important to the PyJNIus authors. In other cases the bridges serve completely different purposes (JPype vs Py4j) and are unlike to ever be consolidated. As far as I can assess from the javabridge documentation this is a much lower level API that JPype. It seems like it focuses on direct JNI calls where the signature has to be specified. It is possible to directly call methods using JPype though most users use the automatic dispatches rather than requiring specifying calls directly. Is is not clear from the JavaBridge documentation if it handles the edge cases such as caller sensitive methods, joining the garbage collectors, array or buffer transfers, or speed of calls. Is this an active project? If not are there features in this that are not available in JPype that need to be added to convert the same space? Are there specialty features that warrant keeping it as a separate project (license, features, size, application space, etc)? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#189>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAFBH5KAUDR266WLAWQOVGTSQ253NANCNFSM4T5E63FQ> .

[Thrameos]

JPype is primarily for scientific code. I develop it for use in radiation and security software for Lawrence Livermore National Laboratory. The main goal is to create Java software which is indistinguishable from Python code to reduce the burden for the Python focused scientists. It thus it uses customizers to make imported Java classes appear as native though you can still access all of the Java methods as well. It is also intended to be bulletproof so that it isn't possible to create a crash through a series of regular operations. I implement outside features as requested so long as they do not compromise performance. As it is not directly funded from government projects, I do it mostly in my free time.

I thought that pyimagej was pyjnius and is currently moving to JPype, imagej/pyimagej#92, though perhaps there is another imagej wrapper out there. I have not contacted CellProfiler group yet to see if what their needs are.

If there are specific features of JavaBridge that would not negatively impact JPype that would make it easier to port to JPype I can place them on the roadmap. If you are using a remote model rather than a JNI, then unfortunately JPype is unlikely to cover the same space as to create ease of use JPype can only have one instance of each class type at a time and thus support only a single JVM.

I believe that JPype covers the same space as Javabridge with regard to numpy, but perhaps if you can point me to some examples I can verify that. JPype is JPype is about the same level of speed as PyJNIus. JPype has some additional overhead because it uses dispatches to delegate rather than calling the method directly. However, as it has caches the dispatch and marshallers, it can run faster in loops and other pieces of critical code. So long as the same argument types are given to a method, it simply verifies the marshallers and skips the resolution stage. As JPype does not automatically convert returns to Python it is much faster than PyJNIus for scientific code where lots of strings or arrays are passed. JPype and PyJNIus are both process linked because they use direct JNI calls. Is this the same with JavaBridge? (that was my guess looking through the source) Or do you use the same model as Py4J in which a remote connection like sockets?

Looking through the Javabridge documentation, I think the most significant difference is the use of javascript in Javabridge to execute code on Java as opposed to JPype in which Python is used to implement Java classes. I am not sure how much of a difficulty that would be to include in JPype.

The other difference being how Java methods are treated. In JPype, you simply request a class and it interrogates the reflection system once to make a new Python class which has all methods available rather and forcing the user to pick and choose methods. There is obviously some burden creating a dynamic Python class though most of this is actually on the Java side which creates a C++ representation for rapid use. There is no JNI-isms exposed to the user but it does expose Java types to Python. Where ever possible, the Java types derive from the base Python type so that user can use them natively without penalty (so a JInt and a Python int appear the same other than JInt having Java restrictions on range and such). The exception being string and arrays which can't exactly be represented due to restrictions in Python API. I could in principle expose Java methods without the dispatch though they would still have marshallers for parameters. (so that things like Python pathlib converts to java.nio.file.Path for example). I am guessing based on your documentation that you always marshall the return so a Java method returning string returns a Python string rather that a Java one, but I may be mistaken. Do you convert strings and arrays on return or leave them on Java side and then mirror the Python API to make them quack (duck type)?

If you can be so kind as to glance over JPype documentation and see if there are other major differences that I would need to address.

The main userguide is pretty long so I would recommend just looking through the quick start guide.

[Thrameos]

Seems like JPype can handle the scripting aspect of run_script without issue.

import jpype
import jpype.imports
from jpype.types import *

jpype.startJVM(classpath="lib/rhino-1.7.13.jar")

def run_script(script, bindings_in = {}, bindings_out = {},
               class_loader = None):
    from org.mozilla.javascript import ContextFactory, ImporterTopLevel, WrappedException
    context = ContextFactory.getGlobal().enterContext()
    sourceName = JString("<java-python-bridge>")
    try :
        if class_loader is not None:
            context.setApplicationClassLoader(class_loader)

        scope = ImporterTopLevel(context)
        for k, v in bindings_in.items():
            scope.put(k, scope, v)
        result = context.evaluateString(scope, script, sourceName, 0, None)
        for k in list(bindings_out):
            bindings_out[k] = scope.get(k, scope)

    except WrappedException as e:
        raise e.getWrappedException()
    finally:
        context.exit()
    return result

Unfortunately, looking at the rest of the javabridge API, it seems there are a very large number of entry points some of which are specialized such as futures. Perhaps not all of them were intended for users, but it it is not clear to me. Thus the cost of converting a javabridge code to JPype would likely be pretty high.

[ctrueden]

@Thrameos @LeeKamentsky Just a heads up: it's on my TODO list to rewrite CellProfiler's Java integration to be built on JPype/scyjava rather than python-javabridge. See imagej/pyimagej#89. I was originally planning to work on this during the month of December, but I've ended up on a leave of absence (mostly) to care for kids till schools reopen. Will keep you guys posted as updates emerge.

Thus the cost of converting a javabridge code to JPype would likely be pretty high.

In general, yeah, but I think in the case of CellProfiler it will be pretty straightforward. It only uses a fraction of the javabridge API.

[Thrameos]

I looked over the API and I think that it may be possible to do most of the wrapper just using customizers we add a name mangler to the class creation process. See jpype-project/jpype#894

As a technical detail the way that JPype works is it has a pile of Java primitives (class, dispatch, field, array) . It sends those to the type constructor in two parts (pre and post). Potentially we can add code in to make it do virtually anything in the pre to rearrange a Java class to change names, hide methods, change methods to properties, even hiding bases, etc.

For the most part your wrapper is just a heavy wrappers which is renaming to Python conventions, applying adapters and converters, and renaming properties.

Here is the current pre where we mangle name conflicts and then apply the customizer overlay. We can generalize this to do much more.

def _jclassPre(name, bases, members):
    # Correct keyword conflicts with Python
    m = list(members.items())
    for k, v in m:
        k2 = pysafe(k)
        if k2 != k:
            del members[k]
            members[k2] = v

    # Apply customizers
    hints = _jcustomizer.getClassHints(name)
    hints.applyCustomizers(name, bases, members)
    return (name, tuple(bases), members)


def _jclassPost(res, *args):
    # Post customizers
    hints = _jcustomizer.getClassHints(res.__name__)
    res._hints = hints
    hints.applyInitializer(res)

    # Attach public inner classes we find
    #   Due to bootstrapping, we must wait until java.lang.Class is defined
    #   before we can access the class structures.
    if _jpype._java_lang_Class:
        for cls in res.class_.getDeclaredClasses():
            if cls.getModifiers() & 1 == 0:
                continue
            wrapper = _jpype.JClass(cls)
            res._customize(str(cls.getSimpleName()), wrapper)