Why there are only _cpu.binarytxt file exist in my module folder after python -m pip install mediapipe

[Terrywang2001]

OS Platform and Distribution

Ubuntu 20.04

Compiler version

No response

Programming Language and version

Python 3.9

Installed using virtualenv? pip? Conda?(if python)

pip

MediaPipe version

0.10.14

Bazel version

No response

XCode and Tulsi versions (if iOS)

No response

Android SDK and NDK versions (if android)

No response

Android AAR (if android)

None

OpenCV version (if running on desktop)

No response

Describe the problem

python -m pip install mediapipe

Complete Logs

I want to use the face_landmarker model on GPU. But after I installed mediapipe, it occurs to me that in /modules/face_landmark folder, the only binarytxt file is face_landmark_front_cpu.binarypb, but shouldn't there be a face_landmark_front_gpu.binarypb file if I want to use solution on GPU? If so, how can I get such file.

Under this circumstance, I followed all the instruction from Face Landmarker Example Code: https://colab.research.google.com/github/googlesamples/mediapipe/blob/main/examples/face_landmarker/python/%5BMediaPipe_Python_Tasks%5D_Face_Landmarker.ipynb. I copied all the necessary code and change the based_option to be the following: base_options = python.BaseOptions(model_asset_path='/absolute/path/face_landmarker.task', delegate=mp.tasks.BaseOptions.Delegate.GPU). But after running this it shows the following error:ValueError: The model is not a valid Flatbuffer buffer. How can I fix this?

[kuaashish]

Hi @Terrywang2001,

It must support the GPU in Ubuntu, To better understand and address the issue, please provide the following:

• The version of MediaPipe you are using.
• The complete standalone code you are using.
• The full error logs.

Thank you!!

[Terrywang2001]

the following is the screenshot of mediapipe/modules/face_landmark folder right after I run "python -m pip install mediapipe":

I don't know why there's no gpu.binarypb file inside this folder, and the same happened for all the folders in the module folder.
.
But the gpu folder still exists.

About the ValueError: The model is not a valid Flatbuffer buffer. I don't know why but it just resolved lol.

[github-actions]

This issue has been marked stale because it has no recent activity since 7 days. It will be closed if no further activity occurs. Thank you.

[Terrywang2001]

So the only code I used is 'python -m pip install mediapipe'. I installed this through my terminal, and when the download finished, only the cpu.binarytxt appeared, but not gpu.binarytxt.

[kuaashish]

Hi @Terrywang2001,

Could you please test this code using a GPU as the delegate? It works fine for me, and I would appreciate knowing if it works for you as well.

Visualization utilities

from mediapipe import solutions
from mediapipe.framework.formats import landmark_pb2
import numpy as np
import matplotlib.pyplot as plt


def draw_landmarks_on_image(rgb_image, detection_result):
  face_landmarks_list = detection_result.face_landmarks
  annotated_image = np.copy(rgb_image)

  # Loop through the detected faces to visualize.
  for idx in range(len(face_landmarks_list)):
    face_landmarks = face_landmarks_list[idx]

    # Draw the face landmarks.
    face_landmarks_proto = landmark_pb2.NormalizedLandmarkList()
    face_landmarks_proto.landmark.extend([
      landmark_pb2.NormalizedLandmark(x=landmark.x, y=landmark.y, z=landmark.z) for landmark in face_landmarks
    ])

    solutions.drawing_utils.draw_landmarks(
        image=annotated_image,
        landmark_list=face_landmarks_proto,
        connections=mp.solutions.face_mesh.FACEMESH_TESSELATION,
        landmark_drawing_spec=None,
        connection_drawing_spec=mp.solutions.drawing_styles
        .get_default_face_mesh_tesselation_style())
    solutions.drawing_utils.draw_landmarks(
        image=annotated_image,
        landmark_list=face_landmarks_proto,
        connections=mp.solutions.face_mesh.FACEMESH_CONTOURS,
        landmark_drawing_spec=None,
        connection_drawing_spec=mp.solutions.drawing_styles
        .get_default_face_mesh_contours_style())
    solutions.drawing_utils.draw_landmarks(
        image=annotated_image,
        landmark_list=face_landmarks_proto,
        connections=mp.solutions.face_mesh.FACEMESH_IRISES,
          landmark_drawing_spec=None,
          connection_drawing_spec=mp.solutions.drawing_styles
          .get_default_face_mesh_iris_connections_style())

  return annotated_image

def plot_face_blendshapes_bar_graph(face_blendshapes):
  # Extract the face blendshapes category names and scores.
  face_blendshapes_names = [face_blendshapes_category.category_name for face_blendshapes_category in face_blendshapes]
  face_blendshapes_scores = [face_blendshapes_category.score for face_blendshapes_category in face_blendshapes]
  # The blendshapes are ordered in decreasing score value.
  face_blendshapes_ranks = range(len(face_blendshapes_names))

  fig, ax = plt.subplots(figsize=(12, 12))
  bar = ax.barh(face_blendshapes_ranks, face_blendshapes_scores, label=[str(x) for x in face_blendshapes_ranks])
  ax.set_yticks(face_blendshapes_ranks, face_blendshapes_names)
  ax.invert_yaxis()

  # Label each bar with values
  for score, patch in zip(face_blendshapes_scores, bar.patches):
    plt.text(patch.get_x() + patch.get_width(), patch.get_y(), f"{score:.4f}", va="top")

  ax.set_xlabel('Score')
  ax.set_title("Face Blendshapes")
  plt.tight_layout()
  plt.show()

Running inference and visualizing the results

import mediapipe as mp
from mediapipe.tasks import python
from mediapipe.tasks.python import vision
import cv2

# STEP 2: Create an FaceLandmarker object.
base_options = python.BaseOptions(model_asset_path='/Users/kuaashish/Downloads/face_landmarker.task', delegate=mp.tasks.BaseOptions.Delegate.GPU)
options = vision.FaceLandmarkerOptions(base_options=base_options,
                                       output_face_blendshapes=True,
                                       output_facial_transformation_matrixes=True,
                                       num_faces=1)
detector = vision.FaceLandmarker.create_from_options(options)

# STEP 3: Load the input image.
image = mp.Image.create_from_file("/Users/kuaashish/Downloads/color_sketch.jpg")

# STEP 4: Detect face landmarks from the input image.
detection_result = detector.detect(image)

# STEP 5: Process the detection result. In this case, visualize it.
annotated_image = draw_landmarks_on_image(image.numpy_view(), detection_result)
cv2.imshow("Annotated_Image",cv2.cvtColor(annotated_image, cv2.COLOR_RGB2BGR))

Please adjust the task file path according to your OS and update the input image path accordingly.

Thank you!!

[Terrywang2001]

It works.

[Terrywang2001]

But still only cpu.binarytxt file exists.

[kuaashish]

Hi @Terrywang2001,

We are currently unaware of this file. Could you please provide the specific use case for this file? Understanding your requirements will help us address the issue more effectively. If your query pertains to running the face landmarker on GPU, it is functioning correctly based on my tests and your comments.

Thank you!!

[Terrywang2001]

Oh it's because I want to use the prebuilt solution.
So in the file "face_detection.py", at the beginning of the file, there are two lines: _SHORT_RANGE_GRAPH_FILE_PATH = 'mediapipe/modules/face_detection/face_detection_short_range_cpu.binarypb'
_FULL_RANGE_GRAPH_FILE_PATH = 'mediapipe/modules/face_detection/face_detection_full_range_cpu.binarypb'.
In it's init def, there's a line about the binary graph path: binary_graph_path = _FULL_RANGE_GRAPH_FILE_PATH if model_selection == 1 else _SHORT_RANGE_GRAPH_FILE_PATH.
So I want to use GPU for the solution, and I thought there should be some file like face_detection_gpu.binarypb. But at this moment, I can only find face_detection_cpu.binarypb. And I can also find face_dection_gpu.pbtxt, but just as I mentioned in the issue #5419 , I couldn't transform pbtxt into binarypb.

[kuaashish]

Hi @Terrywang2001,

It appears you are still using our legacy face detection model instead of the new one, so there is no option to switch between the short-range and full-range models. We recommend using our new Task Face Landmarker API as example provided above. If you are missing a specific file, it may be because you need to build from source to obtain the gpu.binarypb file. You can try using a command similar to the one below:

bazel build -c opt --define MEDIAPIPE_DISABLE_GPU=1 //mediapipe/examples/desktop/face_detection:face_detection_gpu

Thank you!!