libprotobuf repeated_field.h CHECK failed: (total_size_) > (0) trying to extract multi_palm_detections proto

[dizcza]

Have I written custom code (as opposed to using a stock example script provided in MediaPipe)

Yes

OS Platform and Distribution

Ubuntu 22.04

MediaPipe Tasks SDK version

legacy tflite models

Task name (e.g. Image classification, Gesture recognition etc.)

HandLandmarks & pose estimation

Programming Language and version (e.g. C++, Python, Java)

C++

Describe the actual behavior

crash when palm score is extracted from proto messages

Describe the expected behaviour

palm score is extracted from proto messages without a crash

Standalone code/steps you may have used to try to get what you need

ASSIGN_OR_RETURN(mediapipe::OutputStreamPoller multi_palm_detections_poller,
                   graph.AddOutputStreamPoller("multi_palm_detections"));
      mediapipe::Packet multi_palm_detections_packet;

        // Palm detection scores.
        // If no detection, set to 1.0 because the palm detector runs on the first frame only.
        float palm_score[] = {1.0f, 1.0f};

        // crash happens somewhere here
        // if I remove this if-statement, everything works fine with either 1 or 2 num_hands
        if (multi_palm_detections_poller.QueueSize() && multi_palm_detections_poller.Next(&multi_palm_detections_packet)) {
          const auto& multi_palm_detections = multi_palm_detections_packet.Get<std::vector<mediapipe::Detection>>();

          for (int palm_id = 0; palm_id < num_hands_found; palm_id++) {
            const mediapipe::Detection& det = multi_palm_detections[palm_id];
            if (det.score_size() > 0) {
              palm_score[palm_id] = det.score(0);
              LOG(INFO) << palm_id << " palm score: " << palm_score[palm_id];
            }
          }

        }

Graph:

# MediaPipe graph that performs hands tracking on desktop with TensorFlow
# Lite on CPU.
# Used in the example in
# mediapipe/examples/desktop/hand_tracking:hand_tracking_cpu.

# CPU image. (ImageFrame)
input_stream: "input_video"

input_stream: "input_video_visualize"

input_side_packet: "NUM_HANDS:num_hands"

# CPU image. (ImageFrame)
output_stream: "output_video"

node {
  calculator: "ConstantSidePacketCalculator"
  output_side_packet: "PACKET:use_prev_landmarks"
  node_options: {
    [type.googleapis.com/mediapipe.ConstantSidePacketCalculatorOptions]: {
      packet { bool_value: true }
    }
  }
}

# Detects/tracks hand landmarks.
node {
  calculator: "HandLandmarkTrackingCpu"
  input_stream: "IMAGE:input_video"
  input_side_packet: "NUM_HANDS:num_hands"
  input_side_packet: "USE_PREV_LANDMARKS:use_prev_landmarks"
  output_stream: "LANDMARKS:landmarks"
  output_stream: "WORLD_LANDMARKS:world_landmarks"
  output_stream: "HANDEDNESS:handedness"
  output_stream: "PALM_DETECTIONS:multi_palm_detections"
  output_stream: "HAND_ROIS_FROM_LANDMARKS:multi_hand_rects"
  output_stream: "HAND_ROIS_FROM_PALM_DETECTIONS:multi_palm_rects"
}

# Subgraph that renders annotations and overlays them on top of the input
# images (see hand_renderer_cpu.pbtxt).
node {
  calculator: "HandRendererSubgraph"
  
  # Change to "input_video" to see the changes in
  # the colorspace (color glove swaps R-G-B channels)
  # input_stream: "IMAGE:input_video"
  input_stream: "IMAGE:input_video_visualize"

  input_stream: "DETECTIONS:multi_palm_detections"
  input_stream: "LANDMARKS:landmarks"
  input_stream: "HANDEDNESS:handedness"
  input_stream: "NORM_RECTS:0:multi_palm_rects"
  input_stream: "NORM_RECTS:1:multi_hand_rects"
  output_stream: "IMAGE:output_video"
}

# need to detect packet presense
node {
  calculator: "PacketPresenceCalculator"
  input_stream: "PACKET:landmarks"
  output_stream: "PRESENCE:landmark_presence"
}

Other info / Complete Logs

I20240306 18:44:07.606578  8018 airtouch_main.cc:778] 0 palm score: 0.6661
[libprotobuf FATAL external/com_google_protobuf/src/google/protobuf/repeated_field.h:353] CHECK failed: (total_size_) > (0):