Skip to content

Latest commit

 

History

History
715 lines (556 loc) · 23.4 KB

install.md

File metadata and controls

715 lines (556 loc) · 23.4 KB
Raw

Installing MediaPipe

Note: To interoperate with OpenCV, OpenCV 3.x and above are preferred. OpenCV 2.x currently works but interoperability support may be deprecated in the future.

Note: If you plan to use TensorFlow calculators and example apps, there is a known issue with gcc and g++ version 6.3 and 7.3. Please use other versions.

Note: To make Mediapipe work with TensorFlow, please set Python 3.7 as the default Python version and install the Python "six" library by running pip3 install --user six.

Choose your operating system:

To build and run Android apps:

To build and run iOS apps:

  • Please see the separate iOS setup documentation.

Installing on Debian and Ubuntu

  1. Checkout MediaPipe repository.

    $ git clone https://github.com/google/mediapipe.git

# Change directory into MediaPipe root directory
$ cd mediapipe

  2. Install Bazel (version between 0.24.1 and 1.2.1).

    Follow the official Bazel documentation to install Bazel manually. Note that MediaPipe doesn't support Bazel 2.0.0+ yet.

  3. Install OpenCV and FFmpeg.

    Option 1. Use package manager tool to install the pre-compiled OpenCV libraries. FFmpeg will be installed via libopencv-video-dev.

    Note: Debian 9 and Ubuntu 16.04 provide OpenCV 2.4.9. You may want to take option 2 or 3 to install OpenCV 3 or above.

    $ sudo apt-get install libopencv-core-dev libopencv-highgui-dev \
                       libopencv-calib3d-dev libopencv-features2d-dev \
                       libopencv-imgproc-dev libopencv-video-dev

    Option 2. Run setup_opencv.sh to automatically build OpenCV from source and modify MediaPipe's OpenCV config.

    Option 3. Follow OpenCV's documentation to manually build OpenCV from source code.

    Note: You may need to modify WORKSPACE and opencv_linux.BUILD to point MediaPipe to your own OpenCV libraries, e.g., if OpenCV 4 is installed in "/usr/local/", you need to update the "linux_opencv" new_local_repository rule in WORKSPACE and "opencv" cc_library rule in opencv_linux.BUILD like the following:

    new_local_repository(
    name = "linux_opencv",
    build_file = "@//third_party:opencv_linux.BUILD",
    path = "/usr/local",
)

cc_library(
    name = "opencv",
    srcs = glob(
        [
            "lib/libopencv_core.so",
            "lib/libopencv_highgui.so",
            "lib/libopencv_imgcodecs.so",
            "lib/libopencv_imgproc.so",
            "lib/libopencv_video.so",
            "lib/libopencv_videoio.so",
        ],
    ),
    hdrs = glob(["include/opencv4/**/*.h*"]),
    includes = ["include/opencv4/"],
    linkstatic = 1,
    visibility = ["//visibility:public"],
)

  4. For running desktop examples on Linux only (not on OS X) with GPU acceleration.

    # Requires a GPU with EGL driver support.
# Can use mesa GPU libraries for desktop, (or Nvidia/AMD equivalent).
sudo apt-get install mesa-common-dev libegl1-mesa-dev libgles2-mesa-dev

# To compile with GPU support, replace
--define MEDIAPIPE_DISABLE_GPU=1
# with
--copt -DMESA_EGL_NO_X11_HEADERS
# when building GPU examples.

  5. Run the Hello World desktop example.

    $ export GLOG_logtostderr=1

# if you are running on Linux desktop with CPU only
$ bazel run --define MEDIAPIPE_DISABLE_GPU=1 \
    mediapipe/examples/desktop/hello_world:hello_world

# If you are running on Linux desktop with GPU support enabled (via mesa drivers)
$ bazel run --copt -DMESA_EGL_NO_X11_HEADERS \
    mediapipe/examples/desktop/hello_world:hello_world

# Should print:
# Hello World!
# Hello World!
# Hello World!
# Hello World!
# Hello World!
# Hello World!
# Hello World!
# Hello World!
# Hello World!
# Hello World!

Installing on CentOS

  1. Checkout MediaPipe repository.

    $ git clone https://github.com/google/mediapipe.git

# Change directory into MediaPipe root directory
$ cd mediapipe

  2. Install Bazel (version between 0.24.1 and 1.2.1).

    Follow the official Bazel documentation to install Bazel manually. Note that MediaPipe doesn't support Bazel 2.0.0+ yet.

  3. Install OpenCV.

    Option 1. Use package manager tool to install the pre-compiled version.

    Note: yum installs OpenCV 2.4.5, which may have an opencv/gstreamer issue.

    $ sudo yum install opencv-devel

    Option 2. Build OpenCV from source code.

    Note: You may need to modify WORKSPACE and opencv_linux.BUILD to point MediaPipe to your own OpenCV libraries, e.g., if OpenCV 4 is installed in "/usr/local/", you need to update the "linux_opencv" new_local_repository rule in WORKSPACE and "opencv" cc_library rule in opencv_linux.BUILD like the following:

    new_local_repository(
    name = "linux_opencv",
    build_file = "@//third_party:opencv_linux.BUILD",
    path = "/usr/local",
)

cc_library(
    name = "opencv",
    srcs = glob(
        [
            "lib/libopencv_core.so",
            "lib/libopencv_highgui.so",
            "lib/libopencv_imgcodecs.so",
            "lib/libopencv_imgproc.so",
            "lib/libopencv_video.so",
            "lib/libopencv_videoio.so",
        ],
    ),
    hdrs = glob(["include/opencv4/**/*.h*"]),
    includes = ["include/opencv4/"],
    linkstatic = 1,
    visibility = ["//visibility:public"],
)

  4. Run the Hello World desktop example.

    $ export GLOG_logtostderr=1
# Need bazel flag 'MEDIAPIPE_DISABLE_GPU=1' if you are running on Linux desktop with CPU only
$ bazel run --define MEDIAPIPE_DISABLE_GPU=1 \
    mediapipe/examples/desktop/hello_world:hello_world

# Should print:
# Hello World!
# Hello World!
# Hello World!
# Hello World!
# Hello World!
# Hello World!
# Hello World!
# Hello World!
# Hello World!
# Hello World!

Installing on macOS

  1. Prework:

    • Install Homebrew.
    • Install Xcode and its Command Line Tools by xcode-select install.

  2. Checkout MediaPipe repository.

    $ git clone https://github.com/google/mediapipe.git

$ cd mediapipe

  3. Install Bazel (version between 0.24.1 and 1.1.0).

    Option 1. Use package manager tool to install Bazel 1.1.0

    # If Bazel 1.1.0+ was installed.
$ brew uninstall bazel
# Install Bazel 1.1.0
$ brew install https://raw.githubusercontent.com/bazelbuild/homebrew-tap/f8a0fa981bcb1784a0d0823e14867b844e94fb3d/Formula/bazel.rb
$ brew link bazel
# Run 'bazel version' to check version of bazel

    Option 2. Follow the official Bazel documentation to install any version of Bazel manually. Note that MediaPipe doesn't support Bazel 1.1.0+ on macOS yet.

  4. Install OpenCV and FFmpeg.

    Option 1. Use HomeBrew package manager tool to install the pre-compiled OpenCV 3.4.5 libraries. FFmpeg will be installed via OpenCV.

    $ brew install opencv@3

# There is a known issue caused by the glog dependency. Uninstall glog.
$ brew uninstall --ignore-dependencies glog

    Option 2. Use MacPorts package manager tool to install the OpenCV libraries.

    $ port install opencv

    Note: when using MacPorts, please edit the WORKSPACE, opencv_macos.BUILD, and ffmpeg_macos.BUILD files like the following:

    new_local_repository(
    name = "macos_opencv",
    build_file = "@//third_party:opencv_macos.BUILD",
    path = "/opt",
)

new_local_repository(
    name = "macos_ffmpeg",
    build_file = "@//third_party:ffmpeg_macos.BUILD",
    path = "/opt",
)

cc_library(
    name = "opencv",
    srcs = glob(
        [
            "local/lib/libopencv_core.dylib",
            "local/lib/libopencv_highgui.dylib",
            "local/lib/libopencv_imgcodecs.dylib",
            "local/lib/libopencv_imgproc.dylib",
            "local/lib/libopencv_video.dylib",
            "local/lib/libopencv_videoio.dylib",
        ],
    ),
    hdrs = glob(["local/include/opencv2/**/*.h*"]),
    includes = ["local/include/"],
    linkstatic = 1,
    visibility = ["//visibility:public"],
)

cc_library(
    name = "libffmpeg",
    srcs = glob(
        [
            "local/lib/libav*.dylib",
        ],
    ),
    hdrs = glob(["local/include/libav*/*.h"]),
    includes = ["local/include/"],
    linkopts = [
        "-lavcodec",
        "-lavformat",
        "-lavutil",
    ],
    linkstatic = 1,
    visibility = ["//visibility:public"],
)

  5. Make sure that Python 3 and Python "six" library is installed.

    $ brew install python
$ sudo ln -s -f /usr/local/bin/python3.7 /usr/local/bin/python
$ python --version
Python 3.7.4
$ pip3 install --user six

  6. Run the Hello World desktop example.

    $ export GLOG_logtostderr=1
# Need bazel flag 'MEDIAPIPE_DISABLE_GPU=1' as desktop GPU is currently not supported
$ bazel run --define MEDIAPIPE_DISABLE_GPU=1 \
    mediapipe/examples/desktop/hello_world:hello_world

# Should print:
# Hello World!
# Hello World!
# Hello World!
# Hello World!
# Hello World!
# Hello World!
# Hello World!
# Hello World!
# Hello World!
# Hello World!

Installing on Windows Subsystem for Linux (WSL)

Note: WSL has historically not provided access to USB cameras. Mediapipe can use a video file as input.

  1. Follow the instruction to install Windows Sysystem for Linux (Ubuntu).

  2. Install Windows ADB and start the ADB server in Windows.

    Note: Window’s and WSL’s adb versions must be the same version, e.g., if WSL has ADB 1.0.39, you need to download the corresponding Windows ADB from here.

  3. Launch WSL.

    Note: All the following steps will be executed in WSL. The Windows directory of the Linux Subsystem can be found in C:\Users\YourUsername\AppData\Local\Packages\CanonicalGroupLimited.UbuntuonWindows_SomeID\LocalState\rootfs\home

  4. Install the needed packages.

    username@DESKTOP-TMVLBJ1:~$ sudo apt-get update && sudo apt-get install -y build-essential git python zip adb openjdk-8-jdk

  5. Install Bazel (version between 0.24.1 and 1.2.1).

    username@DESKTOP-TMVLBJ1:~$ curl -sLO --retry 5 --retry-max-time 10 \
https://storage.googleapis.com/bazel/0.27.0/release/bazel-0.27.0-installer-linux-x86_64.sh && \
sudo mkdir -p /usr/local/bazel/0.27.0 && \
chmod 755 bazel-0.27.0-installer-linux-x86_64.sh && \
sudo ./bazel-0.27.0-installer-linux-x86_64.sh --prefix=/usr/local/bazel/0.27.0 && \
source /usr/local/bazel/0.27.0/lib/bazel/bin/bazel-complete.bash

username@DESKTOP-TMVLBJ1:~$ /usr/local/bazel/0.27.0/lib/bazel/bin/bazel version && \
alias bazel='/usr/local/bazel/0.27.0/lib/bazel/bin/bazel'

  6. Checkout MediaPipe repository.

    username@DESKTOP-TMVLBJ1:~$ git clone https://github.com/google/mediapipe.git

username@DESKTOP-TMVLBJ1:~$ cd mediapipe

  7. Install OpenCV and FFmpeg.

    Option 1. Use package manager tool to install the pre-compiled OpenCV libraries. FFmpeg will be installed via libopencv-video-dev.

    username@DESKTOP-TMVLBJ1:~/mediapipe$ sudo apt-get install libopencv-core-dev libopencv-highgui-dev \
                       libopencv-calib3d-dev libopencv-features2d-dev \
                       libopencv-imgproc-dev libopencv-video-dev

    Option 2. Run setup_opencv.sh to automatically build OpenCV from source and modify MediaPipe's OpenCV config.

    Option 3. Follow OpenCV's documentation to manually build OpenCV from source code.

    Note: You may need to modify WORKSPACE and opencv_linux.BUILD to point MediaPipe to your own OpenCV libraries, e.g., if OpenCV 4 is installed in "/usr/local/", you need to update the "linux_opencv" new_local_repository rule in WORKSPACE and "opencv" cc_library rule in opencv_linux.BUILD like the following:

    new_local_repository(
    name = "linux_opencv",
    build_file = "@//third_party:opencv_linux.BUILD",
    path = "/usr/local",
)

cc_library(
    name = "opencv",
    srcs = glob(
        [
            "lib/libopencv_core.so",
            "lib/libopencv_highgui.so",
            "lib/libopencv_imgcodecs.so",
            "lib/libopencv_imgproc.so",
            "lib/libopencv_video.so",
            "lib/libopencv_videoio.so",
        ],
    ),
    hdrs = glob(["include/opencv4/**/*.h*"]),
    includes = ["include/opencv4/"],
    linkstatic = 1,
    visibility = ["//visibility:public"],
)

  8. Run the Hello World desktop example.

    username@DESKTOP-TMVLBJ1:~/mediapipe$ export GLOG_logtostderr=1

# Need bazel flag 'MEDIAPIPE_DISABLE_GPU=1' as desktop GPU is currently not supported
username@DESKTOP-TMVLBJ1:~/mediapipe$ bazel run --define MEDIAPIPE_DISABLE_GPU=1 \
    mediapipe/examples/desktop/hello_world:hello_world

# Should print:
# Hello World!
# Hello World!
# Hello World!
# Hello World!
# Hello World!
# Hello World!
# Hello World!
# Hello World!
# Hello World!
# Hello World!

Installing using Docker

This will use a Docker image that will isolate mediapipe's installation from the rest of the system.

  1. Install Docker on your host system.

  2. Build a docker image with tag "mediapipe".

    $ git clone https://github.com/google/mediapipe.git
$ cd mediapipe
$ docker build --tag=mediapipe .

# Should print:
# Sending build context to Docker daemon  147.8MB
# Step 1/9 : FROM ubuntu:latest
# latest: Pulling from library/ubuntu
# 6abc03819f3e: Pull complete
# 05731e63f211: Pull complete
# ........
# See http://bazel.build/docs/getting-started.html to start a new project!
# Removing intermediate container 82901b5e79fa
# ---> f5d5f402071b
# Step 9/9 : COPY . /mediapipe/
# ---> a95c212089c5
# Successfully built a95c212089c5
# Successfully tagged mediapipe:latest

  3. Run the Hello World desktop example.

    $ docker run -it --name mediapipe mediapipe:latest

root@bca08b91ff63:/mediapipe# GLOG_logtostderr=1 bazel run --define MEDIAPIPE_DISABLE_GPU=1 mediapipe/examples/desktop/hello_world:hello_world

# Should print:
# Hello World!
# Hello World!
# Hello World!
# Hello World!
# Hello World!
# Hello World!
# Hello World!
# Hello World!
# Hello World!
# Hello World!

  4. Build Mediapipe Android demos.

    $ docker run -it --name mediapipe mediapipe:latest

root@bca08b91ff63:/mediapipe# bash ./setup_android_sdk_and_ndk.sh

# Should print:
# Android NDK is now installed. Consider setting $ANDROID_NDK_HOME environment variable to be /root/Android/Sdk/ndk-bundle/android-ndk-r18b
# Set android_ndk_repository and android_sdk_repository in WORKSPACE
# Done

root@bca08b91ff63:/mediapipe# bazel build -c opt --config=android_arm64 mediapipe/examples/android/src/java/com/google/mediapipe/apps/objectdetectiongpu:objectdetectiongpu

# Should print:
# Target //mediapipe/examples/android/src/java/com/google/mediapipe/apps/objectdetectiongpu:objectdetectiongpu up-to-date:
# bazel-bin/mediapipe/examples/android/src/java/com/google/mediapipe/apps/objectdetectiongpu/objectdetectiongpu_deploy.jar
# bazel-bin/mediapipe/examples/android/src/java/com/google/mediapipe/apps/objectdetectiongpu/objectdetectiongpu_unsigned.apk
# bazel-bin/mediapipe/examples/android/src/java/com/google/mediapipe/apps/objectdetectiongpu/objectdetectiongpu.apk
# INFO: Elapsed time: 144.462s, Critical Path: 79.47s
# INFO: 1958 processes: 1 local, 1863 processwrapper-sandbox, 94 worker.
# INFO: Build completed successfully, 2028 total actions

Setting up Android SDK and NDK

Requirements:

  • Android SDK release 28.0.3 and above.
  • Android NDK r17c and above.

MediaPipe recommends setting up Android SDK and NDK via Android Studio, and see next section for Android Studio setup. However, if you prefer using MediaPipe without Android Studio, please run setup_android_sdk_and_ndk.sh to download and setup Android SDK and NDK before building any Android example apps.

If Android SDK and NDK are already installed (e.g., by Android Studio), set $ANDROID_HOME and $ANDROID_NDK_HOME to point to the installed SDK and NDK.

export ANDROID_HOME=<path to the Android SDK>
export ANDROID_NDK_HOME=<path to the Android NDK>

In order to use MediaPipe on earlier Android versions, MediaPipe needs to switch to a lower Android API level. You can achieve this by specifying api_level = <api level integer> in android_ndk_repository() and/or android_sdk_repository() in the WORKSPACE file.

Please verify all the necessary packages are installed.

  • Android SDK Platform API Level 28 or 29
  • Android SDK Build-Tools 28 or 29
  • Android SDK Platform-Tools 28 or 29
  • Android SDK Tools 26.1.1
  • Android NDK 17c or above

Using MediaPipe with Gradle

MediaPipe can be used within an existing project, such as a Gradle project, using the MediaPipe AAR target defined in mediapipe_aar.bzl. Please see the separate MediaPipe Android Archive Library documentation.

Using MediaPipe with Bazel

The MediaPipe project can be imported to Android Studio using the Bazel plugins. This allows the MediaPipe examples and demos to be built and modified in Android Studio. To incorporate MediaPipe into an existing Android Studio project, see: "Using MediaPipe with Gradle". The steps below use Android Studio 3.5 to build and install a MediaPipe example app.

  1. Install and launch Android Studio 3.5.

  2. Select Configure | SDK Manager | SDK Platforms.

    • Verify that Android SDK Platform API Level 28 or 29 is installed.
    • Take note of the Android SDK Location, e.g., /usr/local/home/Android/Sdk.

  3. Select Configure | SDK Manager | SDK Tools.

    • Verify that Android SDK Build-Tools 28 or 29 is installed.
    • Verify that Android SDK Platform-Tools 28 or 29 is installed.
    • Verify that Android SDK Tools 26.1.1 is installed.
    • Verify that Android NDK 17c or above is installed.
    • Take note of the Android NDK Location, e.g., /usr/local/home/Android/Sdk/ndk-bundle or /usr/local/home/Android/Sdk/ndk/20.0.5594570.

  4. Set environment variables $ANDROID_HOME and $ANDROID_NDK_HOME to point to the installed SDK and NDK.

    export ANDROID_HOME=/usr/local/home/Android/Sdk

# If the NDK libraries are installed by a previous version of Android Studio, do
export ANDROID_NDK_HOME=/usr/local/home/Android/Sdk/ndk-bundle
# If the NDK libraries are installed by Android Studio 3.5, do
export ANDROID_NDK_HOME=/usr/local/home/Android/Sdk/ndk/<version number>

  5. Select Configure | Plugins install Bazel.

  6. On Linux, select File | Settings| Bazel settings. On macos, select Android Studio | Preferences | Bazel settings. Then, modify Bazel binary location to be the same as the output of $ which bazel.

  7. Select Import Bazel Project.

    • Select Workspace: /path/to/mediapipe and select Next.
    • Select Generate from BUILD file: /path/to/mediapipe/BUILD and select Next.
    • Modify Project View to be the following and select Finish.
    directories:
  # read project settings, e.g., .bazelrc
  .
  -mediapipe/objc
  -mediapipe/examples/ios

targets:
  //mediapipe/examples/android/...:all
  //mediapipe/java/...:all

android_sdk_platform: android-29

  8. Select Bazel | Sync | Sync project with Build files.

    Note: Even after doing step 4, if you still see the error: "no such package '@androidsdk//': Either the path attribute of android_sdk_repository or the ANDROID_HOME environment variable must be set.", please modify the WORKSPACE file to point to your SDK and NDK library locations, as below:

    android_sdk_repository(
    name = "androidsdk",
    path = "/path/to/android/sdk"
)

android_ndk_repository(
    name = "androidndk",
    path = "/path/to/android/ndk"
)

  9. Connect an Android device to the workstation.

  10. Select Run... | Edit Configurations....

    • Select Templates | Bazel Command.
    • Enter Target Expression: //mediapipe/examples/android/src/java/com/google/mediapipe/apps/facedetectioncpu
    • Enter Bazel command: mobile-install.
    • Enter Bazel flags: -c opt --config=android_arm64.
    • Press the [+] button to add the new configuration.
    • Select Run to run the example app on the connected Android device.