VTT in MP4: samples with multiple VTTCue box not rendered correctly

[kqyang]

NOTE: For bugs, if you delete this template, we will send it again and ask you to fill it out.

Have you read the FAQ and checked for duplicate open issues?:

Yes

What version of Shaka Player are you using?:

I am using the public demo app.

Can you reproduce the issue with our latest release version?:

Yes.

Can you reproduce the issue with the latest code from master?:

Yes.

Are you using the demo app or your own custom app?:

demo app.

If custom app, can you reproduce the issue using our demo app?:

What browser and OS are you using?:

Chrome on Linux.

What are the manifest and license server URIs?:

Segmented VTT in MP4:
https://storage.googleapis.com/wvtemp/kqyang/vtt5/bunny_live.mpd

VTT in text file:
https://storage.googleapis.com/wvtemp/kqyang/vtt5/bunny_vod.mpd

(NOTE: you can send the URIs to shaka-player-issues@google.com instead, but please use GitHub and the template for the rest)
(NOTE: a copy of the manifest text or an attached manifest will not be enough to reproduce your issue, and we will ask you to send a URI instead)

What did you do?

Play: https://shaka-player-demo.appspot.com/demo/#asset=https://storage.googleapis.com/wvtemp/kqyang/vtt5/bunny_live.mpd;lang=en-US;build=uncompiled

What did you expect to happen?

Render "WebVtt testing Line 1 (1.0 - 8.5)" and "WebVtt testing Line 2 (1.0 - 8.5)" from 1s to 8.5s and render "WebVtt testing (5.0 - 12.5)" from 5s to 12.5s.

What actually happened?

Seeing "WebVtt testing Line 1 (1.0 - 8.5)" from 1s to 8.5s and "WebVtt testing Line 2 (1.0 - 8.5)" from 8.5s to 12.5s.

The text version works. The streams work in dash.js as well although with line 1 and line 2 reversed.

Here is the contents of the VTT file in text:

WEBVTT

1
00:00:01.000 --> 00:00:08.500
WebVtt testing Line 1 (1.0 - 8.5)

2
00:00:01.000 --> 00:00:08.500
WebVtt testing Line 2 (1.0 - 8.5)

3
00:00:05.000 --> 00:00:12.500
WebVtt testing (5.0 - 12.5)

[TheModMaker]

It looks like the MP4 text segment contains a trun box that contains 3 samples but there are 6 vttc boxes. I have only looked at the spec a little, but it seems that they should be the same and we assume that they are. If that is not the case, how are we supposed to map the vttc cue to a sample time?

[kqyang]

Yes, exactly, the cues in the same sample does have identical time. Right now, the second cue is not shown in Shaka Player.

[TheModMaker]

So the problem is that we just parse the mdat as a box structure and then associate the vttc boxes we see with the trun. What we should be doing is using the trun to find the samples and then parse just the sample. Then we should treat multiple vttc in the sample as having the same time.

[joeyparrish]

Fix cherry-picked to v2.3.5.