Presenting frames in the correct time

[luthfiampas]

Hello, I would like to thank you for this awesome project, I've learned a lot from this!

I'm fairly new to FFmpeg programming, in the tutorial project, you are using Thread.Sleep() to simulate video FPS. However I still don't understand how did you manage to display/present each frame in the correct time?

I'm aware there's pts field in AVFrame. But I have very little idea on how to use this properly. So my current attempt is by utilizing StopWatch to present frame which is very ugly, CPU intensive and inaccurate.

// Wait until stopwatch reaches position
var position = frame.pts * av_q2d(stream.time_base) * 1000; // in ms
while (sw.ElapsedMilliseconds < position)
{
}

DisplayFrame();

Any assistance / explanation you can provide would be greatly appreciated!

[SuRGeoNix]

Hey @luthfiampas

You are on the right track. What is the best way to 'trigger' an event to present the frame... There is a lot of space to answer this question. How am I doing it in Flyleaf...

I've the Screamer thread for this reason. It is similar to the one that you have above with one exception.... It waits to fire both video/audio/subs in a single thread.... and most importantly it doesn't spins until the time comes... but it sleeps (min time required to present video frame, or audio/subs frame). Thread.Sleep will calm down the CPU and will allow other threads to run. What is the best way? Not sure, but if you look at the most common open source players out there you will see similarities.

Let's take ffplay as another reference:

You will see the read_thread that it does what my demuxer/decoder threads doing. Then you will see refresh_loop_wait_event / video_refresh / video_display which is the similar with my Screamer and PresentFrame. refresh_loop_wait_event is mainly what you are asking, which is uses av_usleep((int64_t)(remaining_time * 1000000.0)); which is defined here

In .NET/windows and with SharpDX I've decided currently to use that way. Might have space for better performance though. It's very important when you write a player to be accurate with the FPS, so you can't avoid a) TimeBeginPeriod(1) to ensure that Thread.Sleep(1) will sleep always 1ms and not more than that... and b) to 'spin/thread.sleep(until pts)' to present the frame accurately. One more consideration on this 'loop' is to ensure that you don't waste a lot of time in every loop so you can present also next frames in time. That's the reason you need to have a solution to drop frames in a case like that. (Imagine if you demux/decode/present frames within the same context/thread, you are probably going to come up with a lot of dropped frames)

Feel free to ask any more clarifications ;)