Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Java interface #5

Closed
ravwojdyla opened this issue Nov 9, 2015 · 112 comments

Comments

@ravwojdyla
Copy link
Contributor

commented Nov 9, 2015

Issue to trace effort of swig interface for java. Started implementation - will update with progress. If anyone has any comments/tips - please feel welcome to join the discussion!

@girving

This comment has been minimized.

Copy link
Contributor

commented Nov 9, 2015

Nice!

@girving

This comment has been minimized.

Copy link
Contributor

commented Nov 9, 2015

Moving this comment here from #3:


There's a testsuite with fairly good converge, but currently it's mostly Python with a few C++ tests. There's also a lot of functionality for building graphs that's currently Python only, in particular the automatic differentiation functionality, though that doesn't matter for evaluation of graphs in Java. There are plans to move this functionality into the underlying C++ in future, at which point Java SWIG bindings would be more useful for creating graphs.

If someone takes up the Java SWIG challenge, we'd be happy to accept it upstream pending review, etc., at which point it would be part of our continuous testing. The details of accepting contributions is in flux at the moment, but that will stabilize.

@pslam

This comment has been minimized.

Copy link

commented Nov 18, 2015

Hello guys
We are also interested in adapting TensorFlow for Java. @ravwojdyla Have you, by any chance, started working on the Swig Interface for Java? If you have, we could join our efforts and collaborate on that

@kylevedder

This comment has been minimized.

Copy link

commented Nov 22, 2015

Hello,
I am working on a SWIG wrap of the main C++ API. You can see my progress thus far on my fork, but what's up there is not finished; I'm currently experiencing a problem in which #include "tensorflow/core/lib/core/error_codes.pb.h" is unable to be resolved and I cannot find the intended file anywhere within the project files. Input of any kind would be greatly appreciated.

@bhack

This comment has been minimized.

Copy link
Contributor

commented Nov 22, 2015

There are javacpp preset available for libiraries like Caffe and OpenCV. See also bytedeco/javacpp-presets#111. Java-cpp enable also IOS with RoboVM

@bhack

This comment has been minimized.

Copy link
Contributor

commented Nov 22, 2015

@bhack

This comment has been minimized.

Copy link
Contributor

commented Nov 22, 2015

/cc @saudet

@ravwojdyla

This comment has been minimized.

Copy link
Contributor Author

commented Nov 22, 2015

@pslam - I was able to work just a little bit on this - could definitely use some help!

@saudet

This comment has been minimized.

Copy link

commented Nov 23, 2015

Hi guys, I believe I have pretty functional bindings for JavaCPP: https://github.com/bytedeco/javacpp-presets/tree/master/tensorflow. Let me know if you see anything that could be done with SWIG, but not JavaCPP. I could definitely use the feedback. (Thanks for the cc @bhack!)

@kylevedder

This comment has been minimized.

Copy link

commented Nov 23, 2015

Very nicely done @saudet! I have almost finished a SWIG wrap, but it seems that your implementation works just as well. I do not see anything that my SWIG wrap can do that yours cannot do. JavaCPP seems very cool, I'll have to look into using it for future projects.

@tngan

This comment has been minimized.

Copy link

commented Nov 23, 2015

Hi @kylevedder, have you resolved the issue related to error_codes.pb.h ?
[Edited]
All .pb.h files are compiled from .proto

@kylevedder

This comment has been minimized.

Copy link

commented Nov 23, 2015

@tngan Yes, that is what I discovered as well. Additionally, the .proto files in this project require ProtoBuff3 to be used. I'm using Ubuntu 14.04 and ProtoBuff3 was not available in my package manager, so I compiled it from source, which I got from the 3.0.0 beta release.

The current roadblock which I am trying to solve is how to get ProtoBuff to recurse over the entire file tree and compile the .proto files into .h and .cc files; doing each folder piecemeal results in failures due to unsatisfied dependencies upon other yet to be compiled .proto files.

@davidzchen

This comment has been minimized.

Copy link
Member

commented Nov 24, 2015

@kylevedder Are your SWIG wrappers in a separate repository or are you working in the tensorflow repository? protoc works similar to other compilers. If you are working in the tensorflow repository or are using Bazel, then you would need to set up the protobuf build targets and the dependencies among them.

If you are working in a separate repository and using a different build system, then you would need to use the protobuf plugin for that build system.

I'd be happy to help you set up the build if you would like.

@kylevedder

This comment has been minimized.

Copy link

commented Nov 24, 2015

@davidzchen Thank you for the offer, any and all help is greatly appreciated.

What I have thus far:

I have already setup Bazel and gotten it to compile into a .whl file, which I then handed over to pip and confirmed that I can run the First TensorFlow program.

I have generated SWIG wrapper files in my forked repository. They are in a folder under core/javaWrapper. [link]

What I am trying to do:

Ultimately, my goal is to generate a .so file which than can be called as a native library in Java. Currently, I'm attempting to use g++ to compile the entire system into a .so file; however, the .proto files need to first be expanded into .hs and .ccs prior to this compilation, and that is what I am trying to do with protoc.

You can see my attempt at a wrap script here to potentially get a better idea of what it is I am getting at, although thus far all of my attempts at using protoc has been directory by directory and, consequently, not in the script.

Finally, any feedback on areas of improvement would be greatly appreciated. Thanks!

@saudet

This comment has been minimized.

Copy link

commented Nov 24, 2015

@kylevedder I already have an .so build as part of the JavaCPP Presets: https://github.com/bytedeco/javacpp-presets/tree/master/tensorflow. Thanks to Bazel, it's really simple. Just apply a patch like this:

diff -ruN tensorflow/tensorflow/cc/BUILD tensorflow-patch/tensorflow/cc/BUILD
--- tensorflow/tensorflow/cc/BUILD  2015-11-22 00:00:02.441829192 +0900
+++ tensorflow-patch/tensorflow/cc/BUILD    2015-11-14 11:15:12.689330351 +0900
@@ -75,6 +75,17 @@
     ],
 )

+cc_binary(
+    name = "libtensorflow.so",
+    copts = tf_copts(),
+    linkshared = 1,
+    deps = [
+        ":cc_ops",
+        "//tensorflow/core:kernels",
+        "//tensorflow/core:tensorflow",
+    ],
+)
+
 filegroup(
     name = "all_files",
     srcs = glob(

And run Bazel like this, for example:

bazel build -c opt //tensorflow/cc:libtensorflow.so

AFAIK, this should gobble up pretty much anything of interest for the C++ API.

@davidzchen

This comment has been minimized.

Copy link
Member

commented Nov 24, 2015

@saudet Is there a reason why you are using a cc_binary rule to build the shared library rather than cc_library? You can just have a cc_library rule with the name tensorflow and the build target will build a shared library called libtensorflow.so.

@kylevedder If your goal is to generate an .so file, then something similar to what @saudet suggested would work.

If you need to use the TensorFlow protos in Java code, then you would need to add dependencies from your java_* Bazel build targets to the proto_library targets that generate the Java classes from the .proto files.

We still have a bit of work to do before we open-source the native proto_library rules (see bazelbuild/bazel#52), but in the meantime, TensorFlow uses the cc_proto_library and py_proto_library rules provided by protobuf, and for Java, you should be able to use the Java genproto rule that is included with Bazel. I will check with the team to find out what the timeline for proto_library is and whether it would be worthwhile to unify the rules provided by Protobuf with genproto.

A few other bits of feedback:

  • I think it would be better to keep the directory names consistent and use java_wrapper rather than javaWrapper
  • Perhaps a better place for the Java wrapper would be //tensorflow/java/wrapper rather than //tensorflow/core/java_wrapper?
  • Internally, we have some build rules that take .swig files and generate the sources. This is more ideal because we would avoid checking in the generated files. I can take a look to see how difficult it would be for us to add some SWIG build rules for Bazel to make stuff like this easier.
@saudet

This comment has been minimized.

Copy link

commented Nov 24, 2015

@davidzchen No reason in particular. I'm new to Bazel and just using linkshared=1 as I've seen mentioned on the mailing list worked. So thanks for the tip! I'll be updating that.

@davidzchen

This comment has been minimized.

Copy link
Member

commented Nov 24, 2015

@saudet Thanks! I was just checking to make sure that it wasn't an issue with Bazel. :) Feel free to let me know or open a bug if you run into any issues.

@kylevedder

This comment has been minimized.

Copy link

commented Nov 24, 2015

@saudet Thanks for the info on using Bazel. I too am new to it and did not realize it was capable of generating a .so in that manner.

@davidzchen Thanks for the addendum about using a cc_library, I modified the example from @saudet accordingly when I implemented my Bazil wrapper build. Also, thank you for the input regarding the directory structure; I have updated my folder structure to align with your suggestions.

Additionally, I was not very clear in my previous comment about generating .so files; while my objective is to generate a .so file from the original source, I also want to include the .cxx file that SWIG generates inside of the .so in order to facilitate the JNI calls. Currently, I'm running into an issue in which I cannot get the SWIG generated .cxx file to compile; it's trying to reference JNI.h, a header located in $JAVA_HOME/include/, but I cannot seem to get Bazel to understand the external include path.

@saudet

This comment has been minimized.

Copy link

commented Nov 24, 2015

@davidzchen Hum, nope, cc_library doesn't work. I don't see any other way to make Bazel pass the -shared option to the compiler: http://bazel.io/docs/be/c-cpp.html.

@davidzchen

This comment has been minimized.

Copy link
Member

commented Nov 24, 2015

@saudet I don't think you need to pass -shared yourself. cc_library should be building a .so by default. Does that work for you?

@davidzchen

This comment has been minimized.

Copy link
Member

commented Nov 24, 2015

@kylevedder You won't be able to add the JNI headers that way since they're outside the workspace. However, Bazel includes the local JDK as a local repository and provides a number of built-in targets (see jdk.WORKSPACE and corresponding jdk.BUILD) you can use to depend on local JDK. These are included in each Bazel workspace by default.

Bazel itself uses JNI and interfaces with the local JDK this way (see src/main/native/BUILD). In this BUILD file, there are two genrules to copy the JNI headers and a cc_library target for the library it is building that uses JNI that depends on the headers, and a includes = ["."] so that the C++ code can include the JNI header with #include <jni.h>. This is currently not documented because we are working on a number of improvements to the external repository mechanism, and the @local-jdk name might change, but we can use it for TensorFlow and any other Bazel project that uses JNI in the meantime.

Here is a patch for your BUILD file that adds the genrule targets for copying the JNI headers you need and some changes to the cc_library target to set up the right dependencies, namely:

  1. Add jni.h and jni_md.h, which are copied to the current package by the genrules to srcs
  2. Add a dependency on //tensorflow/core so that you can include the headers under tensorflow/core/public. Note that, headers or any source file in a separate directory are in a separate package from Bazel's point of view, and you will need to add a dependency on the build target that contains those files.
diff --git a/tensorflow/core/java/wrapper/BUILD b/tensorflow/core/java/wrapper/BUILD
index 72b4076..04a3394 100644
--- a/tensorflow/core/java/wrapper/BUILD
+++ b/tensorflow/core/java/wrapper/BUILD
@@ -7,10 +7,30 @@ exports_files(["LICENSE"])
 load("/tensorflow/tensorflow", "tf_copts")
 load("/tensorflow/tensorflow", "tf_gen_op_wrappers_cc")

+genrule(
+    name = "copy_link_jni_md_header",
+    srcs = ["//external:jni_md_header-linux"],
+    outs = ["jni_md.h"],
+    cmd = "cp -f $< $@",
+)
+
+genrule(
+    name = "copy_link_jni_header",
+    srcs = ["//external:jni_header"],
+    outs = ["jni.h"],
+    cmd = "cp -f $< $@",
+)
+
 cc_library(
     name = "java_wrapper",
-    srcs = glob(["*.cc","*.cxx","*.h"]),
-    copts = ["-I$$JAVA_HOME/include/", "-I$$JAVA_HOME/include/linux/"],
+    srcs = glob(["*.cc", "*.cxx", "*.h"]) + [
+        ":jni.h",
+        ":jni_md.h",
+    ],
+    includes = ["."],
+    deps = [
+        "//tensorflow/core",
+    ],
     visibility = ["//visibility:public"],
 )

Note that in general, compile actions in Bazel are run from the root of the source tree, and you would need to change the includes in your SWIG file as follows and then re-generate the C++ files so that they will have the correct includes as well:

diff --git a/tensorflow/core/java/wrapper/tensor_c_api.i b/tensorflow/core/java/wrapper/tensor_c_api.i
index d08b571..9ab1fa1 100644
--- a/tensorflow/core/java/wrapper/tensor_c_api.i
+++ b/tensorflow/core/java/wrapper/tensor_c_api.i
@@ -1,8 +1,8 @@
 %module tensor_c_api_module
 %{
-#include "../../public/tensor_c_api.h"
+#include "tensorflow/core/public/tensor_c_api.h"
 %}
-%include "../../public/tensor_c_api.h"
+%include "tensorflow/core/public/tensor_c_api.h"
 %include "stddef.h"

Once this works, you would have the JNI build set up for Linux since the copy_link_jni_md_header genrule only copies the Linux-specific header. To have it copy the correct platform-specific JNI header, we would need to do the following:

  1. Set up cpu config_settings for other platforms. Currently, tensorflow has a config_setting for --cpu=darwin in tensorflow/python/BUILD. We should probably move that a more appropriate package such as //tensorflow/core. Basically, we would want the same set of config_settings as Bazel (see src/BUILD).
  2. Have copy_link_jni_md_header copy the right JNI header based on which config setting is set using select(), similar to the one in Bazel. Our genrule would look something like the following:
genrule(
    name = "copy_link_jni_md_header",
    srcs = select({
        "//tensorflow/core:darwin": ["//external:jni_md_header-darwin"],
        "//tensorflow/core:darwin_x86_64": ["//external:jni_md_header-darwin"],
        "//tensorflow/core:freebsd": ["//external:jni_md_header-freebsd"],
        "//conditions:default": ["//external:jni_md_header-linux"],
    }),
    outs = ["jni_md.h"],
    cmd = "cp -f $< $@",
)

I'd be happy to help you with this if you run into any issues. Let me know if this works for you.

@saudet

This comment has been minimized.

Copy link

commented Nov 25, 2015

@davidzchen cc_library generates a bunch of .a files, but no .so file. I'm using 0.1.0 as was previously recommended for TensorFlow... Maybe it's fixed in 0.1.1? I'll have to try again.

@kylevedder

This comment has been minimized.

Copy link

commented Nov 25, 2015

@davidzchen Thank you very much for your help. I have followed your instructions and updated both the Java wrapper BUILD file as well as the SWIG .i file as you suggested. Additionally, I moved the wrap script from core/java/wrapper to the root directory and updated the links accordingly.

For now, I have skipped the generalization the genrule for the jni_md.h file, instead focusing on trying to get libtensorflow.so built. Unfortunately, it appears to me as though libtensorflow.so is not being generated; I ended up searching my entire file system for anything named some variant of "libtensorflow" and nothing relevant appeared. It may be named differently or this may be a simple case of user error. Additionally, there is a possibility that it may be related to the issue that @saudet is experiencing with the cc_library rule for .so generation.

Once again, thank you for all of your help, I really appreciate it.

@davidzchen

This comment has been minimized.

Copy link
Member

commented Nov 25, 2015

Sorry, it turns out I was wrong. In order to build a .so that includes the transitive dependencies, what @saudet did using cc_binary with linkshared = 1 and name = "libtensorflow.so" was correct. From the cc_binary.linkshared documentation:

Create a shared library. To enable this attribute, include linkshared=1 in your rule. By default this option is off. If you enable it, you must name your binary libfoo.so (or whatever is the naming convention of libraries on the target platform) for some sensible value of foo.

The main difference between the .so's built by cc_library targets and the .so built with cc_binary using the method described above is that the cc_library artifacts only contain the code in srcs. This is why building cc_library targets with no srcs and only deps, such as //tensorflow/core, do not produce any artifacts. On the other hand, cc_binary targets will link in all the transitive dependencies.

I apologize for the confusion. Perhaps we should improve our documentation and add an example on building .sos.

@ivanseidel

This comment has been minimized.

Copy link

commented Nov 26, 2015

I guess you should follow those steps to build Tensorflow and all it's dependencies. We are working on porting TensorFlow to node.js, and I've implemented a shell script to compile and getter only essential sources from the whole repo:
https://github.com/node-tensorflow/node-tensorflow/blob/1.0.0/tools/install.sh#L233-L282

@bergwerf

This comment has been minimized.

Copy link

commented Mar 28, 2018

What is the state of training models in Java? I have been thinking of writing an ImageJ (popular and free image analysis suite) plugin to apply approaches such as https://arxiv.org/pdf/1505.04597.pdf (recently wildly popular in image segmentation for cell tracking and biomedical applications). I think it would be useful to provide a range of pre-trained models and enable users to refine these for their specific use case. I have been looking into DL4J for this purpose. Are there any concrete plans to allow fitting in the TF Java bindings?

@asimshankar

This comment has been minimized.

Copy link
Contributor

commented Mar 28, 2018

@bergwerf : Training in Java is certainly possible, if not particularly convenient.
You can find a sample at https://github.com/tensorflow/models/tree/master/samples/languages/java/training

(Also, I'm sure you're aware, but see also https://imagej.net/TensorFlow)

@bergwerf

This comment has been minimized.

Copy link

commented Mar 28, 2018

@loretoparisi

This comment has been minimized.

Copy link

commented Mar 29, 2018

@asimshankar that's awesome 👍 💯 🥇 , I'm going to add to my repo https://github.com/loretoparisi/tensorflow-java

whchung referenced this issue in ROCmSoftwarePlatform/tensorflow-upstream Apr 10, 2018
Rbiessy pushed a commit to Rbiessy/tensorflow that referenced this issue Jun 15, 2018
@ghost ghost referenced this issue Nov 28, 2018
darthsuogles pushed a commit to darthsuogles/tensorflow that referenced this issue Dec 8, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
You can’t perform that action at this time.