How to build ARM CPU plugin

Introduction

OpenVINO™ ARM CPU plugin is not included in Intel® Distribution of OpenVINO™. To use the plugin, it should be built from source code.

Building for Linux

Approach #1: build OpenCV, OpenVINO™ and the plugin using pre-configured Dockerfile (cross-compiling, the preferred way)

OpenVINO™ and ARM CPU plugin could be built in Docker* container for 32-bit Debian* Stretch, Buster and 64-bit Ubuntu*:

Clone openvino_contrib repository:

git clone --recurse-submodules --single-branch --branch=master https://github.com/openvinotoolkit/openvino_contrib.git

Go to ARM CPU plugin directory:

cd openvino_contrib/modules/arm_plugin

Build a Docker* image:

docker image build -t arm-plugin -f dockerfiles/Dockerfile.RPi32_buster .

Build the plugin in Docker* container:

Build process is performed by /arm_cpu_plugin_build.sh script executed inside /arm_cpu_plugin directory (default container command to execute). All intermediate results and build artifacts are stored inside working directory.

So one could either mount the whole working directory to get all results stored outside of the container:

mkdir build
docker container run --rm -ti -v $PWD/build:/arm_cpu_plugin arm-plugin

or export only the archive with artifacts:

docker container run --rm -ti --tmpfs /arm_cpu_plugin:rw -v $PWD:/remote \
                     arm-plugin sh -c "sh /arm_cpu_plugin_build.sh && cp ./OV_ARM_package.tar.gz /remote"

NOTE: There are a few environment variables that control /arm_cpu_plugin_build.sh script execution.

BUILD_JOBS controls number of threads for simultaneous compilation

BUILD_TYPE controls Debug/Release configuration (Release by default)

UPDATE_SOURCES controls sources retrievement

clean - don't reload sources if already loaded, just clean build folders (default)

reload - delete all loaded sources and retrieve them again

check - don't reload or cleanup sources

WITH_OMZ_DEMO builds C++ Open Model Zoo demos (ON by default)

Extract the archive to target ARM platform

Approach #2: build OpenVINO™ and the plugin without OpenCV using Docker* (cross-compiling)

Clone openvino and openvino_contrib repositories:

git clone --recurse-submodules --single-branch --branch=master https://github.com/openvinotoolkit/openvino.git 
git clone --recurse-submodules --single-branch --branch=master https://github.com/openvinotoolkit/openvino_contrib.git

Build ie_cross_armhf image by performing the 3rd and the 4th steps of [the guideline].
Run Docker* container with mounted both openvino and openvino_contrib repositories:

docker container run --rm -it -v /absolute/path/to/openvino:/openvino \
                     -v /absolute/path/to/openvino_contrib:/openvino_contrib ie_cross_armhf /bin/bash

The next commands of the procedure need to be run in ie_cross_armhf container.
4. Install scons in the container:

apt-get install scons

Go to openvino directory:

cd openvino

Prepare a build folder:

mkdir build && cd build

Build OpenVINO™ with ARM plugin:

 cmake -DCMAKE_BUILD_TYPE=Release \
       -DCMAKE_TOOLCHAIN_FILE="../cmake/arm.toolchain.cmake" \
       -DTHREADING=SEQ \
       -DIE_EXTRA_MODULES=/openvino_contrib/modules \
       -DBUILD_java_api=OFF .. && make

As soon as make command is finished you can find the resulting OpenVINO™ binaries in the openvino/bin/armv7l/Release and the plugin libarmPlugin.so in openvino/bin/armv7l/Release/lib.

Approach #3: build OpenVINO™ and the plugin without OpenCV on ARM platform (native compiling)

In order to build the plugin, you must prebuild OpenVINO™ package from source using this guideline.

Afterwards plugin build procedure is as following:

Install necessary build dependencies:

sudo apt-get update  
sudo apt-get install -y git cmake  scons build-essential

Clone openvino_contrib repository:

git clone --recurse-submodules --single-branch --branch=master https://github.com/openvinotoolkit/openvino_contrib.git

Go to plugin directory:

cd openvino_contrib/modules/arm_plugin

Prepare a build folder:

mkdir build && cd build

Build plugin:

cmake -DInferenceEngineDeveloperPackage_DIR=<path to OpenVINO package build folder> -DCMAKE_BUILD_TYPE=Release .. && make

NOTE: Use -DARM_COMPUTE_SCONS_JOBS=N cmake flag to speed up the building of ArmCompute library via N simultaneously executed build jobs.

Building for macOS

NOTE: macOS build procedure has been validated on Apple Mac M1 with macOS 11.1 only.

Clone openvino and openvino_contrib repositories:

git clone --recurse-submodules --shallow-submodules --depth 1 --branch=master https://github.com/openvinotoolkit/openvino.git && \
git clone --recurse-submodules --shallow-submodules --depth 1 --branch=master https://github.com/openvinotoolkit/openvino_contrib.git

Save build directory:

root_dir=$(pwd)

There is an issue in the current ACL release which causes build errors on M1. To avoid this issue the specific ACL patch should be applied:

cd openvino_contrib/modules/arm_plugin/thirdparty/ComputeLibrary && \
wget "https://review.mlplatform.org/changes/ml%2FComputeLibrary~6706/revisions/4/patch?zip" -O patch.zip && \
unzip patch.zip && \
git apply 48f2615.diff

Configure Python environment:

cd $root_dir && \
python3 -m pip install --upgrade pip && \
pip3 install virtualenv && \
virtualenv -ppython3 venv && \
source venv/bin/activate && \
pip3 install pybind11

Build OpenVINO and ARM plugin:

mkdir ie_build ?? \
cd ie_build ?? \
cmake -G Ninja \
      -DCMAKE_BUILD_TYPE=Release \
      -DENABLE_BEH_TESTS=OFF \
      -DENABLE_CLDNN=OFF \
      -DENABLE_FUNCTIONAL_TESTS=OFF \
      -DENABLE_MKL_DNN=ON \
      -DENABLE_TESTS=OFF \
      -DENABLE_SAMPLES=ON \
      -DENABLE_PYTHON=ON \
      -DENABLE_INTEL_MYRIAD=OFF \
      -DCMAKE_INSTALL_PREFIX=$root_dir/install_ie \
      -DPYTHON_EXECUTABLE=$root_dir/venv/bin/python \
      -DTHREADING=SEQ \
      -DIE_EXTRA_MODULES=$root_dir/openvino_contrib/modules/arm_plugin \
      -DARM_COMPUTE_SCONS_JOBS=4 \
      -DPYTHON_LIBRARY=/Library/Frameworks/Python.framework/Versions/Current/lib \
      $root_dir/openvino && \
cmake --build . -- && \
ninja install

The final package of OpenVINO with ARM plugin is available at $root_dir/install_ie

Known limitations on macOS

Operations with FP16 data are not supported: ACL#944

Building for Android OS

Create work directory:

mkdir openvino_android
export WORK_DIR="$(pwd)/openvino_android"

Clone openvino and openvino_contrib repositories:

git clone --recurse https://github.com/openvinotoolkit/openvino.git "$WORK_DIR/openvino"
git clone --recurse https://github.com/openvinotoolkit/openvino_contrib.git "$WORK_DIR/openvino_contrib"

Download Android NDK and set environment for it

mkdir "$WORK_DIR/android-tools"
wget https://dl.google.com/android/repository/commandlinetools-linux-7583922_latest.zip
unzip commandlinetools-linux-7583922_latest.zip
yes | ./cmdline-tools/bin/sdkmanager --sdk_root="$WORK_DIR/android-tools" --licenses
./cmdline-tools/bin/sdkmanager --sdk_root="$WORK_DIR/android-tools" --install "ndk-bundle"

Build OpenVINO and ARM plugin for ARM64:

mkdir "$WORK_DIR/openvino_build" "$WORK_DIR/openvino_install"
cmake -GNinja \
      -DVERBOSE_BUILD=ON \
      -DCMAKE_BUILD_TYPE=Release \
      -DCMAKE_TOOLCHAIN_FILE="$WORK_DIR/android-tools/ndk-bundle/build/cmake/android.toolchain.cmake" \
      -DANDROID_ABI=arm64-v8a \
      -DANDROID_STL=c++_shared \
      -DANDROID_PLATFORM=29 \
      -DENABLE_SAMPLES=ON \
      -DENABLE_INTEL_MYRIAD=OFF \
      -DENABLE_INTEL_MYRIAD_COMMON=OFF \
      -DBUILD_java_api=ON \
      -DTHREADING=SEQ \
      -DIE_EXTRA_MODULES="$WORK_DIR/openvino_contrib/modules" \
      -DARM_COMPUTE_SCONS_JOBS=$(nproc) \
      -DCMAKE_INSTALL_PREFIX="$WORK_DIR/openvino_install" \
      -B "$WORK_DIR/openvino_build" -S "$WORK_DIR/openvino"
ninja -C "$WORK_DIR/openvino_build"
ninja -C "$WORK_DIR/openvino_build" install

The libraries and binaries of OpenVINO with ARM plugin on Android is available at $WORK_DIR/openvino_install

For create Android application (Android Studio):

Please find *.so libraries for jniLibs/arm64-v8a at $WORK_DIR/openvino_install/runtime/lib/aarch64
Please find *.jar libraries for app/libs at $WORK_DIR/openvino_install/runtime/lib/aarch64
Please find libc++_shared.so at $WORK_DIR/android-tools/ndk-bundle/sources/cxx-stl/llvm-libc++/libs/arm64-v8a/libc++_shared.so

Provide feedback

Saved searches

Use saved searches to filter your results more quickly