Failed to run Android demo targeting API < 20

[hamidb]

I tried to build the demo application for Android for KitKat device (API 20). I was able to modify the app not to use new hardware.camera2 and it worked on Android Lollipops successfully. However there is an issue targeting older Android device.
It turned out that the app crashes right after trying to load the native shared library, tensorflow_demo in my case.
And that's due to dependency of libpthread.so to libmediandk.so that is not supported for API older than 21. See the following logcat message:

Error: ..dlopen failed: could not load library "libmediandk.so" needed by "libtensorflow_demo.so"; caused by library "libmediandk.so" not found

I also desperately tried to replace the cloned libpthread.so with the same file from my /usr/arm-linux-gnueabi/lib/ directory with no success.
I am using:
1- android-ndk-r10c
2- api_level=20
3- build_tools_version = "20.0.0"

Did I missed anything in that process?

[andrewharp]

Hi Hamid,

The libpthread.so target is just a kludge to get around the fact that the
protobuf library attempts to link it in, despite this not being necessary
(or possible) on Android.

A cleaner workaround for you is to edit google/protobuf/BUILD and change
the line:

LINK_OPTS = ["-lpthread"]
to:
LINK_OPTS = [""]

Ideally the protobuf linkopts would be dependent on build target, or the
dummy libpthread.so file could be generated at compile-time so that it
would match the target architecture (will look into it).

On Sat, Dec 5, 2015 at 5:19 PM, Hamid Bazargani notifications@github.com
wrote:

I tried to build the demo application for Android for KitKat device (API
20). I was able to modify the app not to use new hardware.camera2 and it
worked on Android Lollipops successfully. However there is an issue
targeting older Android device.
It turned out that the app crashes right after trying to load the native
shared library, tensorflow_demo in my case.
And that's due to dependency of libpthread.so to libmediandk.so that is
not supported for API older than 21. See the following logcat message:

Error: ..dlopen failed: could not load library "libmediandk.so" needed by
"libtensorflow_demo.so"; caused by library "libmediandk.so" not found

I also desperately tried to replace the cloned libpthread.so with the
same file from my /usr/arm-linux-gnueabi/lib/ directory with no success.
I am using:
1- android-ndk-r10c
2- api_level=20
3- build_tools_version = "20.0.0"

Did I missed anything in that process?

—
Reply to this email directly or view it on GitHub
#419.

[hamidb]

Hi Andrew,
I think pthread also resolves some POSIX symbols like getPageSize, strtof, ... that are used elsewhere.
By removing -lpthread, I still get unresolved reference errors.
How should I resolve them by removing pthread ?
Thanks,
Hamid

[andrewharp]

The libpthread.so in the TF repo is not actually a copy of libpthead.so
(it's actually arbitrarily cloned from libmediandk.so, which could explain
your earlier link errors), so it's not possible that it could be providing
anything. Those functions are found in other libraries on Android.
-DHAVE_PTHREAD still applies, it just doesn't need to actually link in
pthread.

If you:

1. remove the "-lpthread" from google/protobuf/BUILD,
2. delete the ":dummy_pthread" dep in tensorflow/examples/android/BUILD
3. build with:
   bazel build tensorflow/examples/android:tensorflow_demo
   --config=android_arm

with no other changes do you still get the same errors? Please attach a log
if so.

On Sat, Dec 5, 2015 at 8:13 PM, Hamid Bazargani notifications@github.com
wrote:

Hi Andrew,
I think pthread also resolves some POSIX symbols like getPageSize, strtof,
... that are used elsewhere.
By removing -lpthread, I still get unresolved reference errors.
How should I resolve them by removing pthread ?
Thanks,
Hamid

—
Reply to this email directly or view it on GitHub
#419 (comment)
.

[hamidb]

First I should mention that there is no issue when trying the app on a Android 5.0 device. The problem happens when using KitKat device.
I followed your instruction. Still no success.
Following is my steps:

1- changed google/protobuf/BUILD

// [""] returns bazel error
LINK_OPTS = []

2- changed tensorflow/examples/android/BUILD

cc_library(
    name = "tensorflow_native_libs",
    srcs = glob(["jni/**/*.cc"]),
    hdrs = glob(["jni/**/*.h"]),
    copts = [
        "-std=c++11",
        "-mfpu=neon",
        "-O2",
    ],
    linkopts = ["-llog -landroid -lm -ljnigraphics"],
    tags = [
        "manual",
        "notap",
    ],
    deps = [
    #    ":dummy_pthread", // commented
        "//tensorflow/core:android_tensorflow_lib",
    ],
)

3- My workspace looks:

android_sdk_repository(
    name = "androidsdk",
    api_level = 20,
    build_tools_version = "20.0.0",
    # Replace with path to Android SDK on your system
    path = "/hamidb/software/TADP/android-sdk-linux",
)
android_ndk_repository(
    name="androidndk",
    path="/hamidb/software/TADP/android-ndk-r10c",
    api_level=21)

4- I issued (I also tried without android_arm)

bazel build --verbose_failures //tensorflow/examples/android:tensorflow_demo --config=android_arm

After running the apk file my logcat logs:

--------- beginning of /dev/log/main
D/Raydium_ts_main( 242): ## Report rate in 10 seconds =1213
I/art ( 1219): GcCauseBackground partial concurrent mark sweep GC freed 6011(335KB) AllocSpace objects, 27(1351KB) LOS objects, 12% free, 6MB/7MB, paused 5.823ms total 48.435ms
--------- beginning of /dev/log/system
I/ActivityManager( 904): START u0 {act=android.intent.action.MAIN cat=[android.intent.category.LAUNCHER] flg=0x10200000 cmp=org.tensorflow.demo/.CameraActivity bnds=[984,1381][1176,1573]} from pid 1219
W/art ( 166): Could not get current activity
I/ActivityManager( 904): Start proc org.tensorflow.demo for activity org.tensorflow.demo/.CameraActivity: pid=11918 uid=10073 gids={50073, 1028, 1015}
W/NvAppProfileService( 904): App Profiles: Enabled
I/PowerServiceCient( 904): Successfully bound to service
_E/art (11918): dlopen("/data/app-lib/org.tensorflow.demo-1/libtensorflow_demo.so", RTLD_LAZY) failed: dlopen failed: cannot locate symbol "getpagesize" referenced by "libtensorflow_demo.so"..._
D/AndroidRuntime(11918): Shutting down VM
E/AndroidRuntime(11918): FATAL EXCEPTION: main
E/AndroidRuntime(11918): Process: org.tensorflow.demo, PID: 11918
E/AndroidRuntime(11918): java.lang.UnsatisfiedLinkError: dlopen failed: cannot locate symbol "getpagesize" referenced by "libtensorflow_demo.so"...
E/AndroidRuntime(11918): at java.lang.Runtime.loadLibrary(Runtime.java:364)
E/AndroidRuntime(11918): at java.lang.System.loadLibrary(System.java:526)
E/AndroidRuntime(11918): at org.tensorflow.demo.TensorflowClassifier.(TensorflowClassifier.java:46)
E/AndroidRuntime(11918): at org.tensorflow.demo.TensorflowImageListener.(TensorflowImageListener.java:51)
E/AndroidRuntime(11918): at org.tensorflow.demo.CameraConnectionFragment.(CameraConnectionFragment.java:212)
E/AndroidRuntime(11918): at org.tensorflow.demo.CameraConnectionFragment.newInstance(CameraConnectionFragment.java:91)
E/AndroidRuntime(11918): at org.tensorflow.demo.CameraActivity.onCreate(CameraActivity.java:30)
E/AndroidRuntime(11918): at android.app.Activity.performCreate(Activity.java:5231)
E/AndroidRuntime(11918): at android.app.Instrumentation.callActivityOnCreate(Instrumentation.java:1087)
E/AndroidRuntime(11918): at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:2160)
E/AndroidRuntime(11918): at android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:2246)
E/AndroidRuntime(11918): at android.app.ActivityThread.access$800(ActivityThread.java:136)
E/AndroidRuntime(11918): at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1197)
E/AndroidRuntime(11918): at android.os.Handler.dispatchMessage(Handler.java:102)
E/AndroidRuntime(11918): at android.os.Looper.loop(Looper.java:136)
E/AndroidRuntime(11918): at android.app.ActivityThread.main(ActivityThread.java:5030)
E/AndroidRuntime(11918): at java.lang.reflect.Method.invoke(Native Method)
E/AndroidRuntime(11918): at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:793)
E/AndroidRuntime(11918): at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:609)
W/ActivityManager( 904): Force finishing activity org.tensorflow.demo/.CameraActivity
W/ActivityManager( 904): Activity pause timeout for ActivityRecord{65595fd8 u0 org.tensorflow.demo/.CameraActivity t73 f}
I/Process (11918): Sending signal. PID: 11918 SIG: 9
W/InputMethodManagerService( 904): Window already focused, ignoring focus gain of: com.android.internal.view.IInputMethodClient$Stub$Proxy@65367aa0 attribute=null, token = android.os.BinderProxy@651cfa98
I/ActivityManager( 904): Process org.tensorflow.demo (pid 11918) has died.
D/Raydium_ts_main( 242): ## Report rate in 10 seconds =1038

[andrewharp]

Thanks Hamid. I'm guessing this might be a discrepancy between what the
compiler expects of API level 21 and what the device provides at level 20.

I've just tried setting my WORKSPACE ndk api_level to 20 and found that the
Android demo still builds. Could you try the same and see if that corrects
the issue?

On Sat, Dec 5, 2015 at 9:33 PM, Hamid Bazargani notifications@github.com
wrote:

I followed your instruction. Still no success.
Following is my steps:
1- changed google/protobuf/BUILD

// [""] returns bazel error
LINK_OPTS = []

2- changed tensorflow/examples/android/BUILD

cc_library(
name = "tensorflow_native_libs",
srcs = glob(["jni//*.cc"]),
hdrs = glob(["jni//*.h"]),
copts = [
"-std=c++11",
"-mfpu=neon",
"-O2",
],
linkopts = ["-llog -landroid -lm -ljnigraphics"],
tags = [
"manual",
"notap",
],
deps = [
# ":dummy_pthread", // commented
"//tensorflow/core:android_tensorflow_lib",
],
)

3- My workspace looks:

android_sdk_repository(
name = "androidsdk",
api_level = 20,
build_tools_version = "20.0.0",
# Replace with path to Android SDK on your system
path = "/hamidb/software/TADP/android-sdk-linux",
)
android_ndk_repository(
name="androidndk",
path="/hamidb/software/TADP/android-ndk-r10c",
api_level=21)

4- I issued (I also tried without android_arm)

bazel build --verbose_failures
//tensorflow/examples/android:tensorflow_demo --config=android_arm
After running the apk file my logcat logs:

--------- beginning of /dev/log/main
D/Raydium_ts_main( 242): ## Report rate in 10 seconds =1213
I/art ( 1219): GcCauseBackground partial concurrent mark sweep GC freed
6011(335KB) AllocSpace objects, 27(1351KB) LOS objects, 12% free, 6MB/7MB,
paused 5.823ms total 48.435ms
--------- beginning of /dev/log/system
I/ActivityManager( 904): START u0 {act=android.intent.action.MAIN
cat=[android.intent.category.LAUNCHER] flg=0x10200000
cmp=org.tensorflow.demo/.CameraActivity bnds=[984,1381][1176,1573]} from
pid 1219
W/art ( 166): Could not get current activity
I/ActivityManager( 904): Start proc org.tensorflow.demo for activity
org.tensorflow.demo/.CameraActivity: pid=11918 uid=10073 gids={50073, 1028,
1015}
W/NvAppProfileService( 904): App Profiles: Enabled
I/PowerServiceCient( 904): Successfully bound to service
E/art (11918):
dlopen("/data/app-lib/org.tensorflow.demo-1/libtensorflow_demo.so",
RTLD_LAZY) failed: dlopen failed: cannot locate symbol "getpagesize"
referenced by "libtensorflow_demo.so"...
D/AndroidRuntime(11918): Shutting down VM
E/AndroidRuntime(11918): FATAL EXCEPTION: main
E/AndroidRuntime(11918): Process: org.tensorflow.demo, PID: 11918
E/AndroidRuntime(11918): java.lang.UnsatisfiedLinkError: dlopen failed:
cannot locate symbol "getpagesize" referenced by "libtensorflow_demo.so"...
E/AndroidRuntime(11918): at java.lang.Runtime.loadLibrary(Runtime.java:364)
E/AndroidRuntime(11918): at java.lang.System.loadLibrary(System.java:526)
E/AndroidRuntime(11918): at
org.tensorflow.demo.TensorflowClassifier.(TensorflowClassifier.java:46)
E/AndroidRuntime(11918): at
org.tensorflow.demo.TensorflowImageListener.(TensorflowImageListener.java:51)
E/AndroidRuntime(11918): at
org.tensorflow.demo.CameraConnectionFragment.(CameraConnectionFragment.java:212)
E/AndroidRuntime(11918): at
org.tensorflow.demo.CameraConnectionFragment.newInstance(CameraConnectionFragment.java:91)
E/AndroidRuntime(11918): at
org.tensorflow.demo.CameraActivity.onCreate(CameraActivity.java:30)
E/AndroidRuntime(11918): at
android.app.Activity.performCreate(Activity.java:5231)
E/AndroidRuntime(11918): at
android.app.Instrumentation.callActivityOnCreate(Instrumentation.java:1087)
E/AndroidRuntime(11918): at
android.app.ActivityThread.performLaunchActivity(ActivityThread.java:2160)
E/AndroidRuntime(11918): at
android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:2246)
E/AndroidRuntime(11918): at
android.app.ActivityThread.access$800(ActivityThread.java:136)
E/AndroidRuntime(11918): at
android.app.ActivityThread$H.handleMessage(ActivityThread.java:1197)
E/AndroidRuntime(11918): at
android.os.Handler.dispatchMessage(Handler.java:102)
E/AndroidRuntime(11918): at android.os.Looper.loop(Looper.java:136)
E/AndroidRuntime(11918): at
android.app.ActivityThread.main(ActivityThread.java:5030)
E/AndroidRuntime(11918): at java.lang.reflect.Method.invoke(Native Method)
E/AndroidRuntime(11918): at
com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:793)
E/AndroidRuntime(11918): at
com.android.internal.os.ZygoteInit.main(ZygoteInit.java:609)
W/ActivityManager( 904): Force finishing activity
org.tensorflow.demo/.CameraActivity
W/ActivityManager( 904): Activity pause timeout for
ActivityRecord{65595fd8 u0 org.tensorflow.demo/.CameraActivity t73 f}
I/Process (11918): Sending signal. PID: 11918 SIG: 9
W/InputMethodManagerService( 904): Window already focused, ignoring focus
gain of: com.android.internal.view.IInputMethodClient$Stub$Proxy@65367aa0
attribute=null, token = android.os.BinderProxy@651cfa9
https://github.com/android.os.BinderProxy/tensorflow/commit/651cfa98
I/ActivityManager( 904): Process org.tensorflow.demo (pid 11918) has died.
D/Raydium_ts_main( 242): ## Report rate in 10 seconds =1038

—
Reply to this email directly or view it on GitHub
#419 (comment)
.

[hamidb]

Thank you Andrew.
I guessed that before and built the demo but never tried after removing pthread from protobus.
Now I confirm that changing to API=20 and removing pthread from both protobus and demo application solved the issue (even without --config=android_arm).

Thanks again for your guidelines.

[andrewharp]

Great, glad that worked!

I have a patch for the original libpthread.so issue coming, so that it will
generate the dummy lib at compile-time and make things more portable.

On Sun, Dec 6, 2015 at 10:03 AM, Vincent Vanhoucke <notifications@github.com

wrote:

Closed #419 #419.

—
Reply to this email directly or view it on GitHub
#419 (comment).

[hamidb]

Thanks Andrew,
The patch can be absolutely useful, especially when POSIX is really needed in an application.

[ghost]

Is this procedure even works for API 19?

[andrewharp]

It should, if you follow these steps:

1. patch CameraConnectionFragment.java to remove the use of the camera2 API and use android.hardware.Camera instead
2. change the API level defined by AndroidManifest.xml
3. set the API level of your NDK appropriately in WORKSPACE

Note that "--config=android_arm" is not necessary or beneficial, and was only included by mistake in earlier comments.

[hamidb]

@CalmWaves
You can checkout my fork from here enabling API<20.

[barantes]

Hi, thanks for the instructions on how to build the library for API < 20.

The steps outlined by @andrewharp don't seem to work with the latest TF code though.

I've modified the Android demo to remove dependencies on APIs lower than 21, modified AndroidManifest.xml, and configured the WORKSPACE file, all that to make the build target API 19.

However, the build fails with the following error:

ERROR: /home/bruno/tensorflow/tensorflow/contrib/android/BUILD:36:1: C++ compilation of rule '//tensorflow/contrib/android:android_tensorflow_inference_jni' failed: linux-sandbox failed: error executing command /home/bruno/.cache/bazel/_bazel_bruno/fc8ca368ddd99b1148acfb9b362d6174/execroot/tensorflow/_bin/linux-sandbox ... (remaining 65 argument(s) skipped).
In file included from tensorflow/contrib/android/jni/jni_utils.cc:32:0:
./tensorflow/contrib/android/jni/limiting_file_input_stream.h: In member function 'virtual int tensorflow::android::LimitingFileInputStream::Read(void*, int)':
./tensorflow/contrib/android/jni/limiting_file_input_stream.h:39:28: error: 'errno' was not declared in this scope
} while (result < 0 && errno == EINTR);
^
./tensorflow/contrib/android/jni/limiting_file_input_stream.h:39:37: error: 'EINTR' was not declared in this scope
} while (result < 0 && errno == EINTR);
^
Target //tensorflow/examples/android:tensorflow_demo failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 1615.615s, Critical Path: 1600.63s

Any advice?

Thanks.

[andrewharp]

@barantes If you add #include <errno.h> to the includes list for tensorflow/contrib/android/jni/limiting_file_input_stream.h, does the build error go away?

[andrewharp]

The include tree for the newer NDK APIs must be including errno.h implicitly. Adding it explicitly fixes the build error on older APIs. Fix incoming.

[barantes]

Hi Andrew. Thanks for the quick response. The library was successfully built after adding the #include <errno.h> to limiting_file_input_stream.h.