Long delay before start of play and pauses throughout video

[crimsonxiii]

Do:

• Used the sample C# and C++ applications
• Play a H.264 video with the force option(s) set to true

See:

• Memory starts to climb (similar to Video playback buffers huge amount of data on W10M #97, I guess) and it takes a few seconds for the video to start playing
• At this point the memory starts going down
• After a few seconds (the buffer time?) the video stops playing and the memory goes up again
• This continues to happen throughout the videos I tested

Remarks:

• A MOV video with a different codec (but which as a result also uses the uncompressedsampleprovider) doesn't have this behavior
• FFPlay of the same version plays the video without problems
• Tested with ffmpeg 3.3.3-1 and 3.4
• Changing the buffertime has an impact in the delay of the pauses, but they still occur

[touseefbsb]

@khouzam I think this issue went unnoticed and needs to be looked at.
@crimsonxiii Do you have any update regarding this issue?

[brabebhin]

Are you by any chance on the latest insider build?

[lukasf]

Your CPU might have problems software decoding the file. FFmpegInterop currently does not use multi threading. One core can easily be overloaded by demanding H264 files. I have seen the same behavior with some of my H264 files. The playback engine will stop playback for a short while until enough samples are queued, then it will play again until out-of-samples.

[crimsonxiii]

@mcosmin222 I guess you mean the Windows 10 version? Windows 10 Pro version 1703 OS build 15063.726. I also tested on a Windows 10 Enterprise version 1709 OS build 16299.64. So these are not the Insider Preview Builds.

@lukasf Could be true. I re-did some of the tests and the MOV is lighter than my test H264. Ffplay for both clips reaches 20% CPU, but FFmpegInterop reaches 100%. I replaced the DLLs of ffplay with the ones I compiled myself and the behavior was fine (still 20%). So I guess it's not ffmpeg that's not multi-threaded, but the calls to avcodec_send_packet and avcodec_receive_frame in FFmpegInterop that are called in a single thread.

[lukasf]

Multithreading is a question of decoder configuration. FFmpeg sure does support multithreaded decoding for H264, it's just that FFmpegInterop does not seem to properly use it.

Do you get 100% CPU utilization on all cores or on one core? If one core is at 100% and all others are very low, then missing multithreading is the problem.

[crimsonxiii]

@lukasf There are short spikes on all cores, but at each given time, probably only one of them goes to 100%.

[lukasf]

Well, that is exactly what you'd expect, when only one core tries to decode the whole video stream (and cannot cope with it).

[crimsonxiii]

@lukasf We actually managed to set thread_count (and added a lock_manager) as proposed in #3 and the issue is gone for certain types of videos (if the bitrate isn't too high).

[lukasf]

@crimsonxiii I have just created a PR #196 to get multi threaded decoding into the official repo.

[lukasf]

When testing playback, be sure to run without debugger attached. For me, all H264 files play well when multithreading is enabled (and no debugger running), even Hi10P files with high bitrate. Only the 4K HEVC clips from my camera with 50Mbit/s will stutter. These will only run smooth with hardware acceleration.

[lukasf]

Adding to that, I'd recommend a very low buffer time of 0-100ms for local file playback. Use bigger buffers only where required, e.g. for streaming. See my latest remark in #97. This will help you a lot with load and seek times.

[touseefbsb]

@lukasf my app is focused only on local file playback, do I need to set buffer rate manually? or is it set in the latest pull request you submitted?

[lukasf]

@touseefbsb Yes you should definitely set the buffer time manually. For some reason, the default buffer time in FFmpegInterop is currently 1 second, which is just way too much for video playback. This will have a huge impact on file open and seek performance, and it will unneccessarily eat a lot of RAM (at least with software decoding). If you ask me for a specific value, I'd actually recommend to use buffer time = 0 for local video file playback. Normal video players do not use any kind of buffering in their decoder filters. For the vast majority of users, this should be the best configuration. You could add an optional setting for people with very slow systems, or maybe users who play from network drives, to use a moderate decoder buffer like 100ms, 200ms, 500ms. With 100ms, file open and seek are still reasonably fast, and RAM usage is also still OK.

[touseefbsb]

@lukasf can I set a value which can be good for both high and low specs systems, maybe 500ms? because if the user has the option to change it themselves I think it wont be such a good experience for them. Also where do I need to set this value? in which project and which file exactly?
Thanks