Skip to content
This repository

mpeg-ts streams drop frames after a while #149

Closed
julianscheel opened this Issue · 30 comments

2 participants

Julian Scheel popcornmix
Julian Scheel

If playing mpeg ts streams (in my case dvb signal streamed through rtsp) omxplayer starts to drop frames after a while (several hours usually).
Before commit aec02f4 this could be triggered by switching audio channels, but now this does not trigger the issue anymore. If the stream starts dropping frames I can't see a way to get out of this state without restarting omxplayer.

popcornmix
Collaborator

If this a local live tv stream? If so what software are you streaming with?
If not, is it an internet URL you are viewing? Is it publically available?

Any way of recording the stream to a file, and having the problem reprodcable from the file?

Julian Scheel

Yes, it's a TV live stream which I stream into my local network. The streaming is done through a FPGA based solution. But let me try to reproduce it with something else.
I was unable to reproduce it when playing from file though. But this might just be due to the fact that I do not have files that are that long.
Do you have a DVB device available for testing? If yes would tvheadend or vlc based streaming be usable for you? I could try to trigger the issue with these.

popcornmix
Collaborator

I have an August DVB-T205, but no external antenna and I get a lot of missing packets due to poor signal.
A reproducable failure (e.g. from a file) would be much easier to debug.

I have a suspicion that timestamp jumps can cause a problem (specifically jumps backwards). Are you able to monitor if that is the case when it fails?

Julian Scheel

@popcornmix This was just what I was checking. It seems to be the "classical" pts wrap issue. (Roughly) once a day the pts on mpeg-ts stream wraps back to 0, this seems not to be properly handled my omxplayer. When playing from a file the beheviour is a little different though. I guess this is because it can consume data as fast as it wants in that case. When the pts wrap happens audio continues as normal, but the image will be played way too fast and a/v sync is lost.
I have uploaded a sample file here: http://jusst.de/files/pts-wrap.ts
It contains mpeg2 ts captured from DVB-S. At about 8 seconds the pts wraps and the image will start to play too fast.

Julian Scheel

A longer version of the clip is also available here: http://jusst.de/files/pts-wrap-big.ts
Has a longer period of playback after the wrap, which makes it easier to see the issue.

popcornmix
Collaborator

Thanks for test clip. I see the problem with omxplayer. xbmc does play the file correctly, so it seems the host app can fix it. Need to find out what they do differently.

Julian Scheel

This seems to be related: xbmc/xbmc#1655

popcornmix
Collaborator

Looks like that patch is in upstream ffmpeg.
https://trac.ffmpeg.org/doxygen/trunk/libavformat_2utils_8c_source.html#l00981
Do we need a newer ffmpeg?

Julian Scheel

I have tried with ffmpeg 0.11.3 but this did not seem to solve the issue for me? Did you try a newer ffmpeg as well?

popcornmix
Collaborator

No, I've not had a chance. Did updated ffmpeg have the patch from xbmc/xbmc#1655?

Julian Scheel

Actually it wasn't... But the git head has it. Unfortunately git head breaks API, so that omxplayer has to be adapted to build against it. Will try to do that today.

Julian Scheel

Ok, I patched omxplayer to work with latest ffmpeg git, see here:
https://gist.github.com/julianscheel/5337656

I link the omxplayer against system ffmpeg. Actually this version of ffmpeg completely breaks a/v sync and lots of frames are dropped on that sample file.

popcornmix
Collaborator

Related to #150?

Julian Scheel

I have merged that continous timestamp patch into ffmpeg 0.11.1, but with that version I still see the same behaviour as without the patch. Have you had any chance to try it out in between?

Julian Scheel

Are audio and video timestamps handled differently in omxplayer? I see that the video timestamps are continous (when printed with -s option), while the audio timestamps wrap back when the pts wrap happens. This does not change whether the patch is applied to ffmpeg or not.

Julian Scheel

Update: When forcing ffmpeg to use the ADD mode for overflow correction it works for the sample file. By default it substracted thed offset boundary, so that there were negative timestamps coming in, which omxplayer seems to dislike.
But live streams are completely broken with this patch. It seems this is due to some breakage of timestamps for the first packets of the audio track.
stream: 1, pts: 0, dts: 0
stream: 1, pts: 2160, dts: 2160
stream: 1, pts: 4320, dts: 4320
stream: 1, pts: 6480, dts: 6480
stream: 1, pts: 8640, dts: 8640
stream: 1, pts: 10800, dts: 10800
stream: 1, pts: 3028775650, dts: 3028775650
stream: 1, pts: 3028777810, dts: 3028777810
stream: 1, pts: 3028779970, dts: 3028779970
stream: 1, pts: 3028782130, dts: 3028782130
stream: 1, pts: 3028784290, dts: 3028784290
stream: 1, pts: 3028786450, dts: 3028786450

The first 6 timestamps seem to be "normalized" to start at zero somehow...

popcornmix
Collaborator

@julianscheel
Can you try https://github.com/huceke/omxplayer/commits/new_stall
I've cleaned up some of the clocking/buffering/seeking/timestamp code and it seems to play your sample better,

Julian Scheel

@popcornmix
Yes, the file seems to play fine with that branch.
What actually seems broken now is playback of live streams. See the log here: http://pastebin.cc/paste/307920d437d4b4ea5643fd6249f76336971ceeb6#dlKRDNQIJEwjTebwP/fB+vW6+1TH66wEdPkv4Pjz/2k=
From that point on nothing happens anymore.

Julian Scheel

I tracked the issue with live streams to the check for m_omx_reader.IsEof() in line 1141 of omxplayer.cpp (https://github.com/huceke/omxplayer/blob/new_stall/omxplayer.cpp#L1141). This fails after the changes. But simply skipping that makes live streams work again. Do we actually need that check in other cases?

popcornmix
Collaborator

I've updated branch (rebase). Can you test if that fixes http/eof bug?

Julian Scheel

@popcornmix
The EOF issue is solved with your rebased branch. But in the new_stall branch seems to be a memory leak. After playing a rtsp stream for some hours omxplayer gets killed by the kernel, as it is running out of memory. I have not seen this on master branch.

popcornmix
Collaborator

I can't think of any change that would cause that. More info would be good:
Are you are running an unmodified omxplayer just once? Are there any (audio) codec changes during the stream?
Can you see the memory being leaked gradually (e.g. though top)? Does any event make it increase (e.g. buffering)?
Can you see it when playing a local file?
Any subs?
Can you grab the log?

Julian Scheel

@popcornmix
Sorry, forget about the memleak. I accidentially wrote the log files into a tmpfs and hence flooded the memory.
But unfortunately the PTS wrap is only handled well on file playback, with streaming playback it does not yet work as expected. To reproduce you can use my provided sample file and vlc as streamer like this:
vlc-wrapper pts-wrap-big.ts :sout=#rtp{sdp=rtsp://:8554/} :sout-all :sout-keep

Then play it with omxplayer:
omxplayer -o hdmi -d rtsp://<serverip>:8554/

The video will stuck at the pts wrap and after a while a few frames are rendered from time to time.

The pts wrap happens at ~7secs after start, so be sure to start omxplayer right after VLC starts streaming.

Julian Scheel

This is the stats output when playing that stream: http://goo.gl/aPJAZ
I am unsure why the RTP errors occur at the PTS wrap.
And this is the omxplayer.log: http://jusst.de/files/omxplayer.log

popcornmix
Collaborator

Thanks, I see the hang using vlc-wrapper. Seems to be stuck in WaitCompletion.
I did improve the logic for this in xbmc recently - I'll try porting that to omxplayer to see if that helps.

Julian Scheel

Great, can you just post a short notify here when there is something to test?

popcornmix
Collaborator

I've pulled in the change (and pushed it out). It improves things a little (when it stalls, keyboard input like q still works), but the underlying stall remains.

I've a suspicion that the jump back in timestamps leaves a video frame with a future timestamp in the decode queue. This is stopping the stream from ending (at least for until that future timestamp is reached).

However I'm struggling to get reproducable behaviour through vlc. Sometimes I get no audio, or other failure modes (depending on excactly when I "join" the stream, I guess).

Is it possible to use another instance of vlc to receive the rtsp stream, and dump it to a file? Ideally this file will misbehave consistently on the Pi.

Julian Scheel

Your issues with joining the stream might be due to the stream analysis in avformat_find_stream_info. I have heavily patched that to work fast for mpeg ts streams. I can upload that patch later today, but it likely breaks other file formats.
I think it should be possible to capture that stream with vlc, I will try that later as well. But saying that I wonder, why the original ts file is playing fine, as it has the same pts jump in it?

popcornmix
Collaborator

I can only assume that vlc is either modifying the timestamps, or buffering audio/video differently.
For example, if vlc batches 5 seconds of video, then 5 seconds of audio, and the GPU fifo sizes are less than 5 seconds, then we will fail to play the stream correctly.

Julian Scheel

I did some more testing. The issue with the vlc played sample seems to be something else. Runnin a real livestream goes ok with PTS wrap with recent ffmpeg versions now. So I close this issue now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.