First-class Java API

[cowwoc]

🚀 Feature

Provide a first-class Java API that exposes PyTorch functionality in a Java-eque manner.

Motivation

The Java ecosystem is huge. There are many developers clamoring to integrate PyTorch pipelines into their workflow but they are unable to do so due to the lack of an official Java API.

Pitch

https://github.com/bytedeco/javacpp-presets/tree/master/pytorch contains a quickly maturing Java binding that is easy to maintain. The author would be more than happy to work with you if you could offer some official backing.

Alternatives

Use a generic python-java bridge (e.g. jpype) to interact with PyTorch. This isn't a good option because many Java developers do not code in Python nor are they interested in doing so.

[skyline75489]

Hi @cowwoc . Thanks for the feedback. According to my research, there's existing PyTorch Java binding as well as Android binding. Is there anything you want to do that's not available in the existing bindings? I don't know the Java world that well to understand why javacpp-presets/pytorch is needed. Could you be kind enough to explain the difference between this project and the existing bindings? Thanks again.

[cowwoc]

@skyline75489 Unless I missed something, it looks like you are talking exclusively about model inference from Java code. I am talking about exposing 100% of the API to Java. Meaning, users should be able to create, training and infer models without needing to interact with any language other than Java. Further, I tried digging into the documentation for the aforementioned bindings and I only saw 5 classes in the Javadoc. Something seems to be missing.

I'll let @saudet elaborate further on https://github.com/bytedeco/javacpp-presets/tree/master/pytorch since he knows it better than I.

[saudet]

I'll let @saudet elaborate further on https://github.com/bytedeco/javacpp-presets/tree/master/pytorch since he knows it better than I.

@skyline75489 Right, how would you port to Java this "simple end-to-end example"?
https://pytorch.org/cppdocs/frontend.html#end-to-end-example

[skyline75489]

Thanks @cowwoc and @saudet for the reply. We at Microsoft are trying to improve PyTorch and help the community. If there’s a clear path towards better Java support inside PyTorch itself, we’ll consider putting some effort in the specific area.

My current understanding: there’s a reasonable amount of JNI bindings inside PyTorch but those are mainly for inferencing. In order to improve the Java support we’ll need to extend/refurbish the Java bindings.

Is my understanding correct? Thanks in advance

[saudet]

Yes, that's correct, roughly speaking the current Java API of PyTorch works for inference, but not for training. However, I was able to map the C++ API with JavaCPP and it works well. Training for MLPs, CNN, RNNs, transformers, etc is doable from Java, see bytedeco/javacpp-presets#623 (comment). The idea would be to write a high-level API on top of that. DJL did that for MXNet using JNA (which works well enough since MXNet's native API is C only, but I've offered to switch to JavaCPP since it's more efficient, see apache/mxnet#19797), while we're already using JavaCPP in the case of DL4J and TF Java since their native C APIs are not fully usable and require some C++. I would propose doing the same for PyTorch.

[gchanan]

We don't have any current plans to officially support full API bindings in java. If there is functionality that makes third-party bindings easier to build/maintain we'd like to hear about that, though.

[Thrameos]

I should mention that JPype has been working on the reverse bridge where Java and Python are integrated in reverse (Java calls Python rather than Python calls Java). I have had a working test branch available for a while called epypj. It is currently stalled because it is a huge number of unit tests required to validate that all valid Python commands can be accessed from Java. That and a dispute with my employer over contributing to Python development have been pretty demotivating, but if others wish to contribute to a fully function CPython in Java which can be used for any Python library (including stuff like numpy and matplotlib) it is doable.

It is basically just a FFI which exposes the CPython API in Java and used JPype so that Java objects properly have a presence in Python space. It does use code generators which use reflection in Python to identify callable structures and modules. These can be "customized" using Java classes to construct Jar files which contain all the stubs required to call Python libraries from Java. Or objects can just be treated as Python generics objects which have a fixed interface which polymorphs to implement Java concepts like Iterable or Container (basically if certain slots are found it automatically adds an interface to class definition of the object being created).

I am not sure if this meets the goal of this project, but you should at least be aware that it exists.

[saudet]

We don't have any current plans to officially support full API bindings in java. If there is functionality that makes third-party bindings easier to build/maintain we'd like to hear about that, though.

Thanks! Generally speaking, anything that makes it easier to bind with tools like pybind11 is welcome. JavaCPP is essentially the equivalent of pybind11 for Java, so anything that makes it easier to bind for Python should also make it easier for Java.

/cc @HGuillemet @jbaron @wmeddie @zaleslaw @Craigacp @karllessard

[zaleslaw]

You are awesome, @saudet Will follow your way here!