-
-
Notifications
You must be signed in to change notification settings - Fork 6.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
AE: fix audio sync #1422
AE: fix audio sync #1422
Conversation
I will test right away. Can you give me a brief rundown o what softAE is/means? Does it just handle pcm or something |
Gave this a try. If I'm right and its for pcm (softAE) I for sure she a difference I think you got it! If so this is huge this bug as been plaguing me for about a year even before the AE merge. Amy plans up updating the rest of are? |
@Albinoman887 |
I'm surprised how little attention this PR is getting its a major bug fix. |
@Albinoman887 |
@FernetMenta - excellent find. Not seeing fluctuations on win32 of more than ~+/-5ms but there have been repeated bug reports of discontinuities while transcoding to AC3 and I'd assumed without getting into it that there were buffering issues in the transcoding code. Will test this asap - win32 and will force trancoding over HDMI - and comment. |
@DDDamian |
@FernetMenta - great - at first glance that was the only obvious issue was the lock. Pulling in now for a walkthru. |
Guys im not an idiot I have helped in development before I guess my my filter of what's OK to post here is skewed but I'm no idiot. I won't post again unless I have a log anymore |
@Albinoman887 problem is that when you say something, everyone even remotely associated with XBMC gets a email notification, use this place for constructive development discussion. stuff you consider a major bug might be the bug most of the people never experience. |
@FernetMenta - I've played with this on win32 and not seen a difference in the stability of the sync error either trancoding or not. There is something very wrong in the trancoding code - among other things ffmpeg returns zero consistently in it's delay value referenced in AEEncoderFFmpeg::GetDelay() so we're getting some very bad fluctuations from that function, leading to the discontinuities when trancoding to AC3 when the total delay goes negative. Digging into that issue. When not trancoding I get rock-solid numbers with or without these changes. When transcoding I get frequent glitches, so digging into that. |
I was thinking about maintaining a counter in the audio engine for the delay instead of accumulating delay over all buffers. When player adds a packet to the engine it increments the counter by the duration of the packet (observe resampling speed). Then transcoding can happen or not, when the audio engine puts the packet into the sink, the counter gets decremented. GetDelay would return the counter + delay of the sink. As far as I can see, currently resampling speed is not taken into account, right? At the beginning of the queue a packet might have a different duration than at the end. |
That makes good logical sense. The resampler will change the number of samples but not the duration as the samplerate should always be factored in - this would be one downside of not tracking it per buffer. Resampling is the obvious case where a packet in / packet out counter would fail - a ratio would be required and could be messy In the case of transcoding with ffmpeg not playing ball reporting its delay we may have to use an alternate arrangement such as simply calculating based on samples sent for transcoding, but I do need to dig deeper there. The frequency of all these calcs (especially doubles) certainly adds processing overhead as they're performed every ~2ms or so by my tests. |
Hmmm..repacking into MLP/MAT frames would be another case where packet-in/packet-out would be cumbersome. |
I thought duration can change as a method of a/v sync. Wrong? I am still measuring a variance of average audio sync error up to 20ms. Even with an increased interval of 2 secs for calculating the average error. |
CAESinkALSA::WaitReady is a c/c of the begin of CAESinkALSA::AddPackets ... |
@FernetMenta - I have a fix for the transcoding issue. It is not currently addressed in this PR, but it fixes the variance you are seeing (specifically with transcoding). Not sure if this PR further tightens sync - doesn't on my set-up as noted above, but I have no objections to this PR if you are seeing further improvement on your set-up. As noted the transcoding issue/fix is different to what this PR addresses. |
Do you have a PR for the transcoding issue? Or is it in your master branch
|
@Albinoman887 - neither, I'm just formatting it a bit for clarity and will post it in good time. It will be available once we determine if it supplements, replaces or is completely independent from this PR, which we are trying to discuss. Pushing it to master will conflict with this PR and vice-versa. This is not a forum for requests etc. You have been asked by three team members to stop - make that four. |
I'm not requesting anything um trying to learn so I can be a dev one day
|
@DDDamian - This PR fixes DTS-HD to AC-3 on my systems. (HDMI 2.0, only AC3 for pass through). I consider it as absolutely required to have those locks, otherwise GetDelay would always return incorrect values. Do you want me to try your fix on my systems? |
@Albinoman887 shut up and observe or learn through other channels. this is the absolute final warning, and i mean it. i have a big fat boot and i will not hesitate a second before using it. if. a single new email with your interferences shows up you are out of here. |
@FernetMenta - that would be great, especially as the scenario you describe would involve DTS>PCM>AC3 with transcoding. My fix at present does not include locking the transcoder stage but has a night-and-day effect. You'll laugh when you see the error. I'm at work right now but will push to my repo tonight with or without the transcoder locking (which should be added regardless, but you'll see the results as-is - will add locking if time allows tonight) and leave you a link here. Cheers! |
@FernetMenta - transcoding delay calcs fixed in DDDamian@55dd5db Main issue found was that in SoftAE::GetDelay() we calculate the duration/delay of m_buffer based on an erroneous frame size - that of the encoded frames of 2ch * 2bytes = 4, when the actual buffer data may be 6ch * 4bytes = 24. This creates an artificial and much longer delay value than what is really stored, in turn leading to discontinuities when PTS/DTS values are calculated by DVDPlayerAudio.cpp. In SoftAE.cpp lines 235-237 I store the framesize and sample rate of the pre-encoded PCM stream to use in the later calcs. There are a few other math issues corrected which lead to bad calcs of the duration stored in the actual encoder buffer, but that's minor compared to the above. Note that I have not added locking of the transcoder stage at this point. Testing on a 5.1 AAC to 5.1 AC3 over HDMI the audio lag as reported in the OSD dropped from -0.020 - 0.100secs range to a nearly rock-steady 39ms +/- 2ms on my setup, which matches the non-transcoding values I typically see. So in terms of actual variance I'm seeing +/-2ms on a steady-state buffer duration of 39ms. The 39ms is the sum of the stream, engine, transcoder and sink buffers. I was not able to test on a 2-ch TV - tests were done on Win 7 x64, quad-core AMD 1.8ghz, ATI 6450 to a Denon receiver over HDMI with transcoding forced by disabling AAC passthrough. Locking the transcoder stage may improve this but I'm doubtful as the result is near identical now with straight passthrough, and the actual duration of the transcoder buffer is small compared to the stream and sink buffers. All in all +/- 2ms is pretty good without more extensive locking which can actually hurt more than help. This removes the extensive discontinuities or glitches when transcoding. |
@DDDamian Currently the average error is calculated every second. Increasing this time to 2 secs smooths the average error and all is fine. Testing on Linux (ALSA) HDMI 2.0, only AC3 as pass through. Testmovie: TrueHD. |
@FernetMenta - glad to hear it - thx for the fast testing! Just curious if you have any sync options enabled when you tested this or if you were going straight of audioclock without any sync/frequency modifying settings. The second test would be whether our combined fixes further improves matters. Will try test both together tonight (if possible to reasonably merge). |
@DDDamian I think I found the reason for the square-wave: Dropped packets were added to the pts output and duplicated packets not. See: 038d1ff3c37b39a5a7fbf5287508bb9c3ca7c316 These 2 patches (038d1ff3c37b39a5a7fbf5287508bb9c3ca7c316 + DDDamian@55dd5db) fix the problem on my systems. |
I wonder if moving m_ptsOutput into Dvdaudio.cpp would end up a smaller |
@elupus |
Give me until tomorrow. If I have not done anything tonight, it's probably |
Okey.. soo: https://github.com/elupus/xbmc/compare/dvdaudio But i'm not sure it solves the issue, since i couldn't really figure out what the result of the old code would be. My analysis tells me that the old code would do the following. First line being the packet entry with b duplicated 3 times:
It so time would stop for a while between the end of the previous packet and the start of the last entered packet. New code does the following:
Ie, time stops for a while after first packet has been entered. Without the last commit i did, it would be:
With time jumping back for each added packet. The new code you posted would do the following i think (xyzåäö is the normal order, ran out of letters).
Here weird things happen. First we stall clock, then clock keeps jumping back, then it skips several frames timestamps. But I can be fully wrong here, it's really hard to deduce :) |
@FernetMenta - that explains a lot - my tests were done with no sync monkeying and I have nigh-perfect sync, and never drop/dupe as I use passthrough formats and am an audio nut lol. Fix confirmed per http://forum.xbmc.org/showthread.php?tid=131237&pid=1200031#pid1200031 @elupus - you make my head explode ;) Can I propose then:
|
@elupus - am I reading it wrong or in CDVDPlayerAudio::OutputPacket() does it not still add the packet when it is supposed to drop it? I see the duplicate, and the normal packet add, but in the skip condition it still writes the packet and just alters the m_skipdupcount? I haven't followed further, just struck me as odd. |
It's more obvious when looking at the other end of the queue: CPTSOutputQueue::Current() It pops of current entry if absolute time >= timestamp. Consider a case where a packet is duplicated a thousand times, this function returns pts of packet +1001. It should pop off an entry when absolute time passes front+1
EDIT: disregard this post, too early in the morning. |
@elupus |
So I've rewritten the pts output queue to not depend on current time, only on current delay from audio output: https://github.com/elupus/xbmc/compare/dvdaudio The feedback on re-added packets should now be immediate on current timestamp, which should be much |
elupus, I have tested your new version. First I was looking only at skip/dupe messages in the log and noticed no difference to your last version. |
Ok, yea i wasn't entirely happy with how it turned out. But the principle |
Combine with elupus's work and rebase? |
I get best results with first 3 commits of https://github.com/elupus/xbmc/compare/dvdaudio and increasing interval for average error calculation (DVDPlayerAudio) from 1 sec to at least 2 seconds.
If elupus includes those lines in his change, this PR is obsolete. |
@elupus |
Feel free to pull those in here yes. |
This will avoid timestamps first advancing, then jumping back which would have been caused by duplicate output at same pts.
updated and rebased. @elupus please hit the button if you are fine with it. |
@elupus |
Yes |
This is a concept fix for a major bug in AE. GetDelay returned random values because it did not consider context switches. In particular this was noticeable when transcoding was active. The audio error in dvdplayer was jumping back and forth by > 100ms.
I have done and tested this only for SoftAE yet but should be easy to add to the other engines too.
Thoughts?