HLS with VTT in MP4: Subtitle rendition not shown during playback

[kqyang]

NOTE: For bugs, if you delete this template, we will send it again and ask you to fill it out.

Have you read the FAQ and checked for duplicate open issues?:

Yes.

What version of Shaka Player are you using?:

Public demo.

Can you reproduce the issue with our latest release version?:

Yes.

Can you reproduce the issue with the latest code from master?:

Yes.

Are you using the demo app or your own custom app?:

Demo app.

If custom app, can you reproduce the issue using our demo app?:

What browser and OS are you using?:

Chrome in Linux.

What are the manifest and license server URIs?:
(NOTE: you can send the URIs to shaka-player-issues@google.com instead, but please use GitHub and the template for the rest)
(NOTE: a copy of the manifest text or an attached manifest will not be enough to reproduce your issue, and we will ask you to send a URI instead)

https://storage.googleapis.com/wvtemp/kqyang/vtt6/bunny_vtt_master.m3u8

What did you do?

Play: https://shaka-player-demo.appspot.com/demo/#asset=https://storage.googleapis.com/wvtemp/kqyang/vtt6/bunny_vtt_master.m3u8;lang=en-US;build=uncompiled

What did you expect to happen?

CC button in seek bar to enable / disable subtitles.

What actually happened?

Not seeing CC button.

It is possible that there is a problem with the content. Let me know where the problem is or why the player does it like it if that is the case.

The player should also display or log an error instead of failing silently.

[TheModMaker]

The problem is that it doesn't have a CODECS attribute. We filter it out since we don't have a registered parser for just MP4 content (without a codec). A fix would be to just add the CODECS. We could try to auto-detect, but is there a way to determine if the MP4 has VTT or TTML?

[joeyparrish]

VTT in MP4 should have a codecs attribute of 'wvtt', right? @kqyang, I believe that should be added if it's missing.

[kqyang]

I actually tried adding wvtt codec: https://shaka-player-demo.appspot.com/demo/#asset=https://storage.googleapis.com/wvtemp/kqyang/vtt11/bunny_vtt_master.m3u8;lang=en-US;build=uncompiled

I am seeing a different error this time: Shaka Error TEXT.INVALID_MP4_VTT.

[TheModMaker]

First I think you put the wvtt codec on the wrong stream. The video stream has a codecs of avc1.42e01e,wvtt, and we don't support multiplexed text+video (although we might be able to once #1266 is fixed). I think you meant to put it on the first #EXT-X-MEDIA stream?

The logs show No init segment for MP4+VTT!. We require an init segment for VTT in MP4 to get the timescale. Looking at your content, it contains a sidx box that contains the timescale, so we could get it from that instead.

[kqyang]

First I think you put the wvtt codec on the wrong stream. The video stream has a codecs of avc1.42e01e,wvtt, and we don't support multiplexed text+video (although we might be able to once #1266 is fixed). I think you meant to put it on the first #EXT-X-MEDIA stream?

#EXT-X-MEDIA does not have CODECS attribute. For HLS, the codec is specified on the variant stream #EXT-X-STREAM-INF. It does not mean that they are multiplexed in the same container.

  CODECS

  The value is a quoted-string containing a comma-separated list of
  formats, where each format specifies a media sample type that is
  present in one or more Renditions specified by the Variant Stream.
  Valid format identifiers are those in the ISO Base Media File
  Format Name Space defined by The 'Codecs' and 'Profiles'
  Parameters for "Bucket" Media Types [RFC6381].

The logs show No init segment for MP4+VTT!. We require an init segment for VTT in MP4 to get the timescale. Looking at your content, it contains a sidx box that contains the timescale, so we could get it from that instead.

There is a init segment specified: #EXT-X-MAP:URI="bunny_text.mp4"

[joeyparrish]

It is tricky to take a list of codecs and figure out which goes to which stream. It is one of the ways in which HLS can be more challenging than DASH. We have a whitelist of audio & video codecs in the HLS parser to help with that. We may need to expand it to text.

[joeyparrish]

Wait, actually, we already have that. Something strange is going on.

[joeyparrish]

Fixed, and cherry-picked to v2.3.3.