Edge browser - Clear stream - Video and Audio being chopped

[KameshDevarakonda]

Have you read the FAQ and checked for duplicate open issues?:
Yes

What version of Shaka Player are you using?:
v2.4.4-uncompiled

Can you reproduce the issue with our latest release version?:
Yes

Can you reproduce the issue with the latest code from master?:

Are you using the demo app or your own custom app?:
Demo App

If custom app, can you reproduce the issue using our demo app?:
NA

What browser and OS are you using?:
Hardware: Microsoft Surface Pro
OS Version details:
OS Edition: Windows 10 Pro
Version: 1803
OS build: 17134.286
Microsoft Edge 42.17134.1.0
Microsoft EdgeHTML 17.17134

What are the manifest and license server URIs?:
https://storage.googleapis.com/truman-vod-dash-alpha-test/r6/live/clear/manifest.mpd

What did you do?
Start the playback and observe the issue at 5 minutes 26 seconds into the stream.

What did you expect to happen?
Smooth playback

What actually happened?
It appears that video frames are being dropped and audio being chopped.

This seems to happen only on Shaka Player on Edge browser on the Windows 10 Pro OS. This doesn't happen on dash.js nightly version on the same browser and OS.

This issue also doesn't happen with Shaka Player on Chrome browser on the Windows 10 Pro OS.

[joeyparrish]

Confirmed. On the transition from period 2 to period 3 at time 5:26, the gap-jumping logic fires and jumps ahead by about 0.4 seconds. This occurs on Edge 17 on the latest nightly build of Shaka Player.

The gap according to the media element is from 327.0066211 to 327.0298412, size 23.22 ms. Going deeper to the MediaSource level, the video buffer has no gap at all. The gap only appears in the audio buffer.

If I effectively disable gap jumping by lowering the "maximum small gap size" setting in the demo, playback is smooth. So the gap detected by the player can safely be ignored.

Let me take some time to compare this to #1435. Both of these issues may require a change to the gap jumping logic.

[joeyparrish]

Actually, I take it back. It appears to be a simple rounding error in the manifest.

In the third period, the first audio sample appears at 327.0298412. The period start is 327 exactly. The gap is caused by the final audio samples in period 2 getting chopped off at 327 exactly, so we lose 29.8412 ms of audio.

So I think the most robust solution might be:

1. Avoid a gap by allowing fudging the end of the append window, so that those last samples don't get cut off on MS Edge.
2. Change gap-jumping logic to avoid jumping very small audio gaps in the first place. Less than 100ms may not be audible, and the browser will likely play through. If the browser does not play through, our stall-detection will still rescue playback.

[kqyang]

In the third period, the first audio sample appears at 327.0298412. The period start is 327 exactly. The gap is caused by the final audio samples in period 2 getting chopped off at 327 exactly, so we lose 29.8412 ms of audio.

The first audio sample of the third period is really at 327.0019, i.e. with a small gap of 0.0019 instead of 0.0298. There might be a bug in Edge not honoring the EditList.

It appears to be a simple rounding error in the manifest.

It is not a rounding error, but packager aligns period timelines with video timestamps based on the assumption that browsers can tolerate a small gap in audio but cannot tolerate a small gap in video.

@joeyparrish Your proposed solution 2 sounds like a good solution to me. Let me know if there is anything we should improve in packager.

[joeyparrish]

I was proposing to do both 1 & 2, actually. But we'll need to discuss more internally.

Right now I'm collecting data on gap tolerance across browsers to help drive the decision.

[joeyparrish]

Other browsers will play through larger gaps, but it seems that Chrome will stall on an audio gap of as little as 20ms. So changing gap-jumping logic may make things worse.

Therefore I think we should just adjust the append window. It fixes this particular piece of content, but we should be careful to make sure it doesn't break anything else.

[kqyang]

Will fudging the end of the append window cause problems for multi-period single file content described in Dash-Industry-Forum/DASH-IF-IOP#166 method 1?

 ... Period 1           |                 Period 2
 ...  | Video Frame N-1 |  Video Frame N | Video Frame N + 1 |  ...

For example, will Video Frame N above be appended twice and cause problems? (Since Period 1 will include part of Video Frame N with end of append window fudging)

[joeyparrish]

I was finally able to verify that fudging the append window does not introduce any problems with any of the demo assets in Shaka Player on any of our supported platforms. @kqyang, if you have any additional content you'd like us to test before we merge this change, please let me know!

[kqyang]

@joeyparrish Yes, I do want to test a few additional contents. I'll let you know when it is ready.

[joeyparrish]

We have tested with several additional assets across browsers to ensure that the fix for this (a fudge factor applied to the append window end) doesn't break anything else.

[joeyparrish]

Fix cherry-picked for v2.4.6.