Crash when processing long-ish video files

[samim23]

When processing longer video files (30s +), i frequently (but somewhat randomly) get a crash & the following error message. When the "--process_real_time" flag is present, the crashes happen always. Running openpose headless in both cases.

command:

./build/examples/openpose/openpose.bin --video media/video.avi --no_display --num_gpu 1 --write_video media/video_out.avi --disable_blending --alpha_pose 0.55

responds:

Starting pose estimation demo.
Starting thread(s)
Empty frame detected. In src/openpose/producer/producer.cpp:checkFrameIntegrity():136
terminate called after throwing an instance of 'std::runtime_error'
what():
Error:
Wrong input element (empty cvInputData).

Coming from:

• src/openpose/core/cvMatToOpInput.cpp:format():21
• src/openpose/core/cvMatToOpInput.cpp:format():46
• ./include/openpose/core/wCvMatToOpInput.hpp:work():76
• ./include/openpose/thread/subThread.hpp:workTWorkers():138
• ./include/openpose/thread/subThreadQueueOut.hpp:work():72
• ./include/openpose/thread/thread.hpp:threadFunction():206

Aborted (core dumped)

[gineshidalgo99]

It does not happen to me with my longest videos (>5min). Since you closed the issue, could I know the problem (and solution)? So other people can benefit if the same error happen to them? Thanks!

[samim23]

The issues i've encountered leading to this "bug" can be backtracked to the ".avi" file format. Been using ffmpeg to convert mp4's into avi's and found one has to be extremely specific in conversion (+ there are many parameters) - as openpose will throw errors on most kinds of files called ".avi". Unclear which settings work reproducable so far. As a workaround, I've ended up simply extracting all frames with ffmpeg into a folder, and then processing those with openpose instead - which works flawless.

It would be fantastic, if openpose besides .avi supported other file I/O formats, such as .MP4.

[gineshidalgo99]

You are right. Right now we are limited by the formats that OpenCV handles, which for video are still limited. We will try in future releases to use other libraries with better ffmpeg support. Thank you for the feed-back!

[samim23]

One more little related request: It would be great if one could define at what framerate openpose creates videos. Useful in the case above, where the frames were split with an external tool.

[gineshidalgo99]

Well, if a external tool was used to split them, the same tool (or any free one) should be able to join them again (e.g. ffmpeg). Making OpenPose doing that would be more error-prone or would require OpenPose to be sure that all the images has the same resolution. Otherwise it will crash with different sizes. So given cons/pros, it is not useful for images.

However, your idea is good for webcam, it might be useful for the webcam in order the generate a video. So I just added the new option camera_fps, so the video will be generated at that FPS for webcam.