Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
..
Failed to load latest commit information.
clock_cycle_profiling
hexagon_controller
hvx_ops_support_checker
README.md

README.md

TensorFlow Runtime with HVX Acceleration

This README explain how to build and use the TensorFlow runtime with HVX Acceleration. HVX is an extension of Hexagon, a DSP provided by Qualcomm, which can compute vector calculations faster using less energy than ARM processors.

Dependencies

  • Android SDK.
  • Android NDK. Save the path in ${NDK_ROOT}.
  • A rooted Qualcomm-based Android device connected to the computer (preferably, a Snapdragon Development Board, but it could be a rooted phone with a Qualcomm SoC, albeit this guide may not work with it). The device needs to be rooted for development and testing purposes, and shouldn't be needed in production. See Behold, The Snapdragon MDP for more information.
  • Hexagon SDK v3.0. Save the path in ${QUALCOMM_SDK}.
  • The current directory should be TensorFlow source code (git clone https://github.com/tensorflow/tensorflow.git && cd tensorflow), and saved into ${TF_ROOT_DIR}.

You may also need to add a test signature in the device to run HVX-based binaries. Follow the instructions in ${QUALCOMM_SDK}/docs/Tools_Signing.html, using Python 2.

Note that if the device is not rooted, you may not be able to get the serial number, push the test signature and/or run binary files that call HVX libraries.

Quick Start Guide

We provide several tools to build and run inference with this runtime quickly.

Run inception model with a prebuilt Hexagon library

If you don’t need to build your own implementation of Hexagon HVX, we provide a shortcut to execute graphs by using pre-compiled binaries.

./tensorflow/contrib/makefile/samples/build_and_run_inception_hexagon.sh -p

The -p option makes the script download dependencies (i.e., Hexagon HVX binaries and graphs models), copy them to the Android device and execute a test.

Run inception model by building all from the source code

If you want to build your own implementation of Hexagon HVX, we provide a sample all-in-one script to execute graphs which downloads the source and builds everything that's necessary.

./tensorflow/contrib/makefile/samples/build_and_run_inception_hexagon.sh

Building libraries

If you've finished walking through the quick start guide, you may want to try building each binary manually.

Build libhexagon_nn_skel.so

Download Hexagon NN library from codeaurora.org and build it. For Hexagon SDK 3.0, we need use the compatible version(721b2d58f) of nnlib.

git clone https://source.codeaurora.org/quic/hexagon_nn/nnlib
cd nnlib
git reset 721b2d58f --hard

Just follow the instructions in README.HOW_TO_BUILD. You can find the file libhexagon_nn_skel.so in hexagon_Release_dynamic_toolv72_v60/ship. Then copy the generated binary to ${GEN_LIBS_DIR}.

GEN_LIBS_DIR="/path/to/a/dir/to/store/hexagon/libraries"
cp -v "hexagon_Release_dynamic_toolv72_v60/ship/libhexagon_nn_skel.so" "${GEN_LIBS_DIR}"

Build libhexagon_controller.so

Download tensorflow and build hexagon controller.

GENERATED_NNLIB_DIRECTORY="/path/to/nnlib"
GENERATED_HEXAGON_CONTROLLER_DIRECTORY="${QUALCOMM_SDK}/examples/common/generated_hexagon_controller"
rm -rf "${GENERATED_HEXAGON_CONTROLLER_DIRECTORY}"
cp -af "${TF_ROOT_DIR}/tensorflow/contrib/hvx/hexagon_controller" \
   "${GENERATED_HEXAGON_CONTROLLER_DIRECTORY}"
cp -afv "${GENERATED_NNLIB_DIRECTORY}/interface" \
"${GENERATED_HEXAGON_CONTROLLER_DIRECTORY}/"
cp -afv "${GENERATED_NNLIB_DIRECTORY}/glue" \
"${GENERATED_HEXAGON_CONTROLLER_DIRECTORY}/"
make clean V=android_Release
rm -rf android_Release
make tree VERBOSE=1 V=android_Release
cp -v "${GENERATED_HEXAGON_CONTROLLER_DIRECTORY}/android_Release/ship/libhexagon_controller.so" "${GEN_LIBS_DIR}"

Build TensorFlow linking Hexagon library

Build TensorFlow with build_all_android.sh specifying the -x option.

BUILD_ALL_ANDROID_PATH="${TF_ROOT_DIR}/tensorflow/contrib/makefile/build_all_android.sh"

CC_PREFIX=${CC_PREFIX} NDK_ROOT=${NDK_ROOT} "${BUILD_ALL_ANDROID_PATH}" \
-x "${GEN_LIBS_DIR}" \
-s "${TF_ROOT_DIR}/tensorflow/contrib/makefile/sub_makefiles/hexagon_graph_execution/Makefile.in" \
-t hexagon_graph_execution

Push binaries to your Android device

Before running tests on your Android device, you need to push several binaries to it.

adb push "${GEN_LIBS_DIR}/libhexagon_controller.so" "/data/local/tmp"
adb push "${GEN_LIBS_DIR}/libhexagon_nn_skel.so" "/vendor/lib/rfsa/adsp"
adb push -p \
"${TF_ROOT_DIR}/tensorflow/contrib/makefile/gen/bin/hexagon_graph_execution" \
"/data/local/tmp/"
adb wait-for-device
ANDROID_EXEC_FILE_MODE=755
adb shell chmod "${ANDROID_EXEC_FILE_MODE}" \
"/data/local/tmp/hexagon_graph_execution"
adb wait-for-device

Run tests on the device

Finally, you can run the inference tests on your device.

adb shell 'LD_LIBRARY_PATH=/data/local/tmp:$LD_LIBRARY_PATH' \
"/data/local/tmp/hexagon_graph_execution"

Troubleshooting

Testsig issue

If you're using the Open-Q 820 Snapdragon Development Kit, you may run into an issue with running the executable due to a missing testsig library. From the Hexagon SDK documentation: Dynamic shared objects are required to be digitally signed and then authenticated at runtime before they are allowed to be loaded and executed. Generating a testsig library is necessary to run the unsigned sample library built from this project.

If the lack of a testsig library is your problem, you will see errors of the type: vendor/qcom/proprietary/adsprpc/src/fastrpc_apps_user.c:169::error: -1: 0 == (nErr = remotectl_open(name, (int*)ph, dlerrstr, sizeof(dlerrstr), &dlerr)) appearing in adb logcat or "Expected: (version) >= (1), actual: 0 vs 1" while running a binary from adb.

You need to add a test signature, as described at the beginning of this README. After rebooting your device, you should be able to run the sample application.

Qualcomm SDK Linux installation fails with "Malformed \uxxxx encoding"

The installation file is based on LaunchAnywhere, which fails in Linux if the PS1 env variable contains non-common Unicode chars:

Preparing to install...
Extracting the JRE from the installer archive...
Unpacking the JRE...
Extracting the installation resources from the installer archive...
Configuring the installer for this system's environment...

Launching installer...

An internal LaunchAnywhere application error has occurred and this application cannot proceed. (LAX)

Stack Trace:
java.lang.IllegalArgumentException: Malformed \uxxxx encoding.
  at java.util.Properties.loadConvert(Properties.java:574)
  at java.util.Properties.load0(Properties.java:391)
  at java.util.Properties.load(Properties.java:317)
  at com.zerog.common.java.util.PropertiesUtil.loadProperties(Unknown Source)
  at com.zerog.lax.LAX.<init>(Unknown Source)
  at com.zerog.lax.LAX.main(Unknown Source)

It can be solved by temporarily assigning the PS1 environment variable to something simple, such as '$'.

Maintainers