TensorFlow Object Detection Demo does not work

[bbongcol]

Hi.
I tried to run the "TensorFlow Object Detection Demo" following the guide.
But the "TensorFlow Object Detection Demo" does not work and only logs related to video decoding are shown as below.

Test Environment

• Ubuntu 16 (Virtual Box)
• OpenCV : Run setup_opencv.sh
• bazel 0.28.1
• Install mediapipe and install related utils following the guide
• Hello World desktop example works ok.

Build object_detection:object_detection_tensorflow
$ bazel build -c opt
--define MEDIAPIPE_DISABLE_GPU=1
--define no_aws_support=true
mediapipe/examples/desktop/object_detection:object_detection_tensorflow

Run object_detection:object_detection_tensorflow
$ export GLOG_logtostderr=1
$ bazel-bin/mediapipe/examples/desktop/object_detection/object_detection_tensorflow
--calculator_graph_config_file=mediapipe/graphs/object_detection/object_detection_desktop_tensorflow_graph.pbtxt
--input_side_packets=input_video_path=input_video_path=mediapipe/examples/desktop/object_detection/test_video.mp4,output_video_path=mediapipe/examples/desktop/object_detection/output.mp4

Output Log
W0827 10:46:38.084439 17235 opencv_video_decoder_calculator.cc:167] Not all the frames are decoded (total frames: 205 vs decoded frames: 2).
E0827 10:46:38.108150 17235 simple_run_graph_main.cc:65] Failed to run the graph: CalculatorGraph::Run() failed in Run:
; Packet timestamp mismatch on a calculator receiving from stream "input_video". Current minimum expected timestamp is 1 but received 0. Are you using a custom InputStreamHandler? Note that some InputStreamHandlers allow timestamps that are not strictly monotonically increasing. See for example the ImmediateInputStreamHandler class comment.
; Packet timestamp mismatch on a calculator receiving from stream "input_video". Current minimum expected timestamp is 1 but received 0. Are you using a custom InputStreamHandler? Note that some InputStreamHandlers allow timestamps that are not strictly monotonically increasing. See for example the ImmediateInputStreamHandler class comment.

[jiuqiant]

bbongcol@, thanks for filing this issue. I reproduced it on Ubuntu 16.04 and figured out the root cause. For some reason, cv::VideoCapture gives back two cv::mat objects with timestamp 0 but the calculator doesn't check if the timestamp of the current frame is greater than the one of the previous frame. Please modify the OpenCVVideoDecoderCalculator like the following. Then, rebuild and rerun the object detection demo.

@ -154,7 +155,10 @@ class OpenCvVideoDecoderCalculator : public CalculatorBase {
                      cv::COLOR_BGRA2RGBA);
       }
     }
-    cc->Outputs().Tag("VIDEO").Add(image_frame.release(), timestamp);
+    if (prev_timestamp_ < timestamp) {
+      cc->Outputs().Tag("VIDEO").Add(image_frame.release(), timestamp);
+      prev_timestamp_ = timestamp;
+    }
     decoded_frames_++;
     return ::mediapipe::OkStatus();
   }
@@ -178,6 +182,7 @@ class OpenCvVideoDecoderCalculator : public CalculatorBase {
   int frame_count_;
   int decoded_frames_ = 0;
   ImageFormat::Format format_;
+  Timestamp prev_timestamp_ = Timestamp::Unset();
 };
 
 REGISTER_CALCULATOR(OpenCvVideoDecoderCalculator);

We will fix this issue soon.

[bbongcol]

Thanks!. It works fine.

I think opencv installation via setup_opencv.sh can be unstable because it always builds the latest source code. So I think it would be better to build with a tested specpic opencv version.

[AItechnology]

@jiuqiant For snowboarding.mp4, it was right, but it got the same error(not all frames decoded 532 vs 531) if I use life_at_google.mp4