Multi-period with presentationTimeOffset can overwrite previous period

[tjber]

Hi,

Continued with [https://groups.google.com/forum/#!topic/shaka-player-users/2RK662rDMk8]

I didn't try to play it with the master.
I have used the demo application of version 2.2.4 in chrome and firefox

I have a system in which I will receive a series of video files containing a number of black frames at the beginning.
Currently the process to play those files is to transcode them to remove those frames and start playing from 0, but I'd like to skip this step.
So, I tried to use the presentationTimeOffset attribute in the mpd file.
<SegmentBase indexRangeExact="true" indexRange="" presentationTimeOffset="" >

The problem is that I need frame accuracy and it seems that this attribute only has second precision.
I have tried several combinations with this attribute and the timeScale without success. Dash especifications says :

presentationTimeOffset : The value of the presentation time offset in seconds is the division of the value of this attribute and the value of the timescale attribute.

but I get the feeling that this attribute is not being taken into account.

I attach the mpd file I'm testing with.
dash.mpd.txt

I have updated in dropbox the files with which the issue can be played back, and I have added a file in which you can see the expected result (expected_file.mp4) https://www.dropbox.com/sh/zfbynu6n9bxhd9q/AAB4TEvA9g71UADgne0pqwKia?dl=0

What I would like to do is to skip the initial frames (marked with a white counter) and that the playback starts in frame marked with the TC 00:00:00:00 of the file with the green TC, after that, in the second period skip the initial frames again and that the playback starts in frame marked with the TC 00:00:00:00 of the file with the red TC.

Currently if I try to jump 3.12 seconds actually a jump of 4 seconds is made, so for the first period the playback starts at TC 00:00:00:22 instead of 00:00:00:00 and the second period does not jump at all.

If I can change some of the manifest file to make this work, any ideas will be welcome.

Thank you in advance.
TJ

[TheModMaker]

First, presentationTimeOffset must be an integer, so the value 3.12 is invalid. You can use the timescale attribute to get decimals, but it should match the timescale in the media. So if the timescale is 100, then the presentationTimeOffset would be 312.

It also looks like we aren't handling the presentationTimeoffset value correctly with timescale. I'm marking this a duplicate of #1099.

[joeyparrish]

@tjber, when #1099 is fixed (soon), you should try again with the presentationTimeOffset field set with respect to timescale. Please follow along on #1099. Sorry for the confusion.

[joeyparrish]

If you are still having issues after #1099 is fixed, we can reopen this issue and try again. Thanks!

[joeyparrish]

The fix for #1099 has been pushed to the master branch.

[tjber]

I'll test it and come back with the results.
Thank you very much!

[tjber]

Hi!
I've tried again, both with the Demo app and the master.

The timescale attribute is already taken into account in the SegmentBase, but frame accuracy is not yet possible, it only jumps from second to second.
I mean, with these values:
<SegmentBase indexRangeExact="true" indexRange="915-1138" timescale = "100" presentationTimeOffset="212" >
Playback starts at second 3, not 2,120.

Also, If the manifest has more than 1 period, the presentationTimeOffset is only taken into account in the first period, playback of the following periods starts in the second 0.

This is the mpd file i used for testing.

dash.mpd.txt

Thanks!

[TheModMaker]

Without a playable URL to test with, it will be hard to diagnose the problem. You can send it to shaka-player-issues@google.com if you can't post it publically.

My guess (without being able to look at the content) is that there is a keyframe at 3 seconds. When we set the presentationTimeOffset, this changes where the media time 0 is. The browser will drop any frames that appear before time 0. If the first frame after 0 is not a keyframe, the browser won't be able to decode it so it will discard those frames until it hits a keyframe.

[tjber]

Hi,

@TheModMaker , I've tried your guess and it's good. I have generated a file in which all the frames are keyframes and so it is positioned correctly.
Unfortunately, in my case it's not a valid solution because I won't be able to regenerate the files, so I'll need to find a workaround.

The playback of the second period still does not take into account the presentationTimeOffset, even with these new files.

I've added the regenerated materials and the two manifests I've tested with to the dropbox folder:
https://www.dropbox.com/sh/zfbynu6n9bxhd9q/AAB4TEvA9g71UADgne0pqwKia?dl=0
dash. mpd --> manifest that I've been doing the tests
dash_allkeyframes. mpd --> manifest that refers to regenerated files in which all frames are keyframes
(you can download and use them freely, they are a replica of my original files that I created to document the issue)

[joeyparrish]

If your goal is to limit what's played by skipping the first, say, 2.1 seconds, you should try the playRangeStart configuration instead of modifying the manifest:

https://shaka-player-demo.appspot.com/docs/api/shakaExtern.html#PlayerConfiguration

player.configure({
  playRangeStart: 2.1
});

Does this help?

[joeyparrish]

Okay, I think I see what's going on now. We are using presentationTimeOffset in the second Period, but the content is being offset back into the previous Period's time range and overwriting content starting at time 11. Period 1's duration is 13.12, PTO for period 2 is -2.12, so the content starts appearing at 11.

[tjber]

Hi @joeyparrish thank you! I´ll try with playRangeStart/playRangeEnd and splitting each period in one manifest, not ideal but quite better than transcode :)

[joeyparrish]

I hope that unblocks you.

In the mean time, though, there does seem to be a bug on our side. We should be instructing MediaSource to throw out any content that hangs off the beginning of the Period, and we are not.

When I start adding that to our code, I still see some strange behavior, and I'm not sure if it's a Chrome bug or a Shaka bug. Still working to track that down.

[joeyparrish]

I've put together a non-Shaka demo that shows MediaSource doing the right thing with this keyframe-only content, so any issues I'm having are issues with Shaka Player.

1098.zip

[joeyparrish]

I've noticed two things that are wrong with your demo content:

1. I think the second period presentationTimeOffset should be 312, not 212
2. The timescale in the manifest does not match the timescale in the media, which is throwing off our SIDX parser

I will need to consult with the DASH spec and IOP to understand whether or not the timescales are supposed to match. If so, the content is wrong. If not, the player is wrong.

[joeyparrish]

The DASH spec says of timescale:

NOTE: This may be any frequency but typically is the media clock frequency of one of the media streams (or a positive integer multiple thereof).

So we should not assume that it matches the timescale of the container.

[joeyparrish]

With the following changes, I have your all-keyframes content working as I think it is intended:

1. Edit the manifest to set presentationTimeOffset in second period to 312 instead of 212
2. Use appendWindowStart in MediaSource to avoid replacing period 1 media with period 2 media
3. Fix segment index parsers such that they do not assume a consistent timescale between media & manifest

You can expect fixes for 2 & 3 to be published in the near future.

[joeyparrish]

I updated the nightly build with the fixes for this. You can now see a demo of your content here:

https://nightly-dot-shaka-player-demo.appspot.com/demo/#asset=//storage.googleapis.com/shaka-demo-assets/_bugs/1098-fixed/dash_allkeyframes.mpd;lang=en-US;play

Is this what you were going for?

[tjber]

Hi @joeyparrish
I'm sorry for the delay in answering. I've been out for a few days.
Thank you very much! that was exactly the behavior I expected.
I'll try to get the "allkeyframe" files sent to me to avoid the initial jump.
Thank you so much.

[joeyparrish]

We are always happy to help. It took some time to understand what was happening, but it turned out to be an important bug. So thank you for the report!

[joeyparrish]

Fixes cherry-picked to v2.2.6.