Tensorflow Lite + Sceneform Android App Example
A simple Android example that demonstrates mixing object detection from camera images with an augmented reality interpretation of the scene so as to draw 3D objects on recognized objects.
Media You Should See
See the slide deck on SlideShare.
See the video of the presentation at
Steps to Build
Note: Steps 1 and 2 are to be run from the main Tensorflow source directory. Do not run commands on this repository until you get to Step 3.
Also note that I have slightly optimized the number of steps required for you to build this project for yourself. As such, it is not exactly the same procedure as described in the video and slides. Specifically, these steps save you from having to build with Gradle before building with Bazel.
Install Java JDK 10. (Not 8, and not 11, possibly until Bazel gets upgraded.) Set up
$JAVA_HOMEas an environment variable in the usual manner, and put
Follow the Bazel steps for the TF Demo App. Ignore the part where it says you can't build Android apps with Bazel on Windows. You can; this repo will show you how.
a. Install Bazel and Android Prerequisites. It's easiest with the latest version of Android Studio.
- You'll need at least SDK version 28. - Also get NDK version 19. - Make sure to install the latest version of Bazel. Some distributions ship with Bazel 0.5.4, which is too old. - Bazel requires Android Build Tools `28.0.3` or higher. - Install the latest version of the Gradle plugin. - You also need to install the Android Support Repository, available through Android Studio under `Android SDK Manager -> SDK Tools -> Android Support Repository`.
b. Edit your
tensorflow/to add SDK and NDK targets.
NOTE: As long as you have the SDK and NDK installed, the
./configurescript will create these rules for you. Answer "Yes" when the script asks to automatically configure the
- Make sure the
WORKSPACEis set to an SDK version that you have installed.
- By default, Android Studio will install the SDK to
~/Android/Sdkand the NDK to
c. If you didn't run
python configure.pyfor Windows users) as indicated in the previous step, run it from your
tensorflow/directory in order to set up the Bazel build.
- Make sure the
If you are on Windows:
a. Install the appropriate Visual C++ SDK. From this EXE (
visualcppbuildtools_full.exe), if you are on Windows 10, choose
Windows 10 SDKand
.NET Framework SDK.
b. Go to
C:\Program Files (x86)\Windows Kits\(version number, e.g.
\. It is possible that the SDK version you installed (likely 10.0.10240) is below another one that is already installed. If this is the case, you will need to copy the the directories underneath
Includeof this directory (especially
um) into the highest version SDK (first moving the originals out of the way by renaming them with a suffix, possibly) so that you will not experience dependency issues during the Bazel build.
Build the Original Source with Bazel
tensorflow/ and run at least
bazel build //tensorflow/lite/examples/android:tflite_demo. A few additional options are listed below, such as configuring the fat_apk_cpu flag to package support for 4 hardware variants and which C++ compiler to use. You may replace it with --config=android_arm64 on a 64-bit device and --config=android_arm for 32-bit device, but the flags are purely optional and are not required for a successful build.
bazel build -c opt --cxxopt='--std=c++11' --fat_apk_cpu=x86,x86_64,arm64-v8a,armeabi-v7a \ //tensorflow/lite/examples/android:tflite_demo
If your Bazel build fails along the way, search Google to try to remedy it. If you find the remedy didn't work, sometimes it is useful to clear the Bazel cache.
Shutdown any running Bazel processes with
Go into your user home directory (such as
C:\Users\me) and delete the entire
Retry your Bazel build.
Include libtensorflow.so dependency into this code
Clone this repository into a location of your choosing. It does not have to be within your Tensorflow source code repository.
libtensorflow.soshared library file on your computer. It is located:
- deep in your user home directory where Bazel stores its temporary files, probably somewhere inside
- in the final APK file that gets built, if you open it as an archive -- probably inside
bazel-bin/or also somewhere in your user directory
- Move this file into the appropriate folder for the architecture for which the library was built -- usually this is
jniLibs/armeabi-v7abut if you pulled it from the APK, you will probably find the name of the architecture from the folder it came from in a similar way as if it came from your user home folder as described above.
Make It Yours
Build the app with Android Studio. Once it installs, you might need to deal with a couple of crashes and force close issues while you grant it permissions to your camera, but upon doing this, you should be able to restart it and it should work just fine. Also if exceptions in the logcat show that it's complaining about not finding files that appear to be in the assets/ directory, just "Rebuild All" and redeploy.
If You Do Anything Cool With This
Let me know so I can give it a try! Easiest way is on Twitter at @SWebCEO.