Can't read webvtt in webm file

[loutoo]

Hi,
I have a webm file. It has an subtitle stream by webvtt format. But Exoplayer can't find it.

THE VIDEO INFO:

Input #0, matroska,webm, from 'a1.webm':
  Metadata:
    ENCODER         : Lavf59.27.100
  Duration: 00:01:59.69, start: -0.006000, bitrate: 2762 kb/s
  Stream #0:0: Video: vp9 (Profile 0), yuv420p(tv, progressive), 1920x1080, SAR 1:1 DAR 16:9, 25 fps, 25 tbr, 1k tbn (default)
    Metadata:
      BPS-eng         : 8169907
      DURATION-eng    : 00:41:52.280000000
      NUMBER_OF_FRAMES-eng: 62807
      NUMBER_OF_BYTES-eng: 2565636839
      _STATISTICS_WRITING_APP-eng: mkvmerge v25.0.0 ('Prog Noir') 64-bit
      _STATISTICS_WRITING_DATE_UTC-eng: 2018-09-07 00:24:14
      _STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
      ENCODER         : Lavc59.37.100 libvpx-vp9
      DURATION        : 00:01:59.120000000
  Stream #0:1(eng): Audio: opus, 48000 Hz, stereo, fltp (default)
    Metadata:
      BPS-eng         : 640000
      DURATION-eng    : 00:41:52.320000000
      NUMBER_OF_FRAMES-eng: 78510
      NUMBER_OF_BYTES-eng: 200985600
      _STATISTICS_WRITING_APP-eng: mkvmerge v25.0.0 ('Prog Noir') 64-bit
      _STATISTICS_WRITING_DATE_UTC-eng: 2018-09-07 00:24:14
      _STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
      ENCODER         : Lavc59.37.100 libopus
      DURATION        : 00:01:59.688000000
  Stream #0:2(eng): Subtitle: webvtt (default)
    Metadata:
      title           : English
      ENCODER         : Lavc59.37.100 webvtt
      DURATION        : 00:01:55.514000000

THE EXOPLAYER LOG:

I/ExoPlayerImpl: Init cf736eb [ExoPlayerLib/2.18.4] [redfin, Pixel 5, Google, 33]
D/EventLogger: playWhenReady [eventTime=0.00, mediaPos=0.00, window=0, true, USER_REQUEST]
D/EventLogger: surfaceSize [eventTime=0.00, mediaPos=0.00, window=0, 0, 0]
D/EventLogger: timeline [eventTime=0.00, mediaPos=0.00, window=0, periodCount=1, windowCount=1, reason=PLAYLIST_CHANGED
D/EventLogger:   period [?]
D/EventLogger:   window [?, seekable=false, dynamic=true]
D/EventLogger: ]
D/EventLogger: mediaItem [eventTime=0.00, mediaPos=0.00, window=0, reason=PLAYLIST_CHANGED]
D/EventLogger: state [eventTime=0.00, mediaPos=0.00, window=0, BUFFERING]
E/OpenGLRenderer: Unable to match the desired swap behavior.
D/EventLogger: surfaceSize [eventTime=0.03, mediaPos=0.00, window=0, 1080, 2138]
E/SurfaceSyncer: Failed to find sync for id=0
W/Parcel: Expecting binder but got null!
D/EventLogger: loading [eventTime=0.03, mediaPos=0.00, window=0, period=0, true]
D/EventLogger: timeline [eventTime=0.03, mediaPos=0.00, window=0, period=0, periodCount=1, windowCount=1, reason=SOURCE_UPDATE
D/EventLogger:   period [?]
D/EventLogger:   window [?, seekable=false, dynamic=false]
D/EventLogger: ]
D/EventLogger: timeline [eventTime=0.33, mediaPos=0.00, window=0, period=0, periodCount=1, windowCount=1, reason=SOURCE_UPDATE
D/EventLogger:   period [2512.33]
D/EventLogger:   window [2512.33, seekable=true, dynamic=false]
D/EventLogger: ]
D/EventLogger: videoEnabled [eventTime=0.33, mediaPos=0.00, window=0, period=0]
D/EventLogger: audioEnabled [eventTime=0.33, mediaPos=0.00, window=0, period=0]
D/EventLogger: tracks [eventTime=0.33, mediaPos=0.00, window=0, period=0
D/EventLogger:   group [
D/EventLogger:     [X] Track:0, id=1, mimeType=video/x-vnd.on2.vp9, res=1920x1080, language=und, selectionFlags=[default], supported=YES
D/EventLogger:   ]
D/EventLogger:   group [
D/EventLogger:     [X] Track:0, id=2, mimeType=audio/opus, channels=2, sample_rate=48000, language=en, selectionFlags=[default], supported=YES
D/EventLogger:   ]
D/EventLogger: ]
I/DMCodecAdapterFactory: Creating an asynchronous MediaCodec adapter for track type video
D/EventLogger: downstreamFormat [eventTime=0.33, mediaPos=0.00, window=0, period=0, id=1, mimeType=video/x-vnd.on2.vp9, res=1920x1080, language=und, selectionFlags=[default]]
D/CCodec: allocate(c2.qti.vp9.decoder)
I/CCodec: setting up 'default' as default (vendor) store
I/CCodec: Created component [c2.qti.vp9.decoder]
D/CCodecConfig: read media type: video/x-vnd.on2.vp9

Exoplayer Version: 2.18.3
Devices: All

Thanks!

[rohitjoins]

Hi,

Thank you for sharing the media file. I tested it on the Exoplayer demo app and observed that the subtitles were successfully displayed.

Please let me know any other details you might have to reproduce this issue.

[loutoo]

I am sorry. I send the wrong video. The mkv is transcoded that I use for testing. I resent the video in question.

[rohitjoins]

Hi @loutoo,

After analyzing the webm file you provided, it appears that there are three available tracks:

The third track in this file contains the WEBVTT subtitle track. The codec ID assigned to this track is D_WEBVTT/SUBTITLES and aligns with http://wiki.webmproject.org/webm-metadata/temporal-metadata/webvtt-in-webm (refer to Storing WebVTT data in a WebM Track).

At present, MatroskaExtractor only supports WEBVTT for .mkv files, with a codec ID defined as S_TEXT/WEBVTT. As a result, Exoplayer does not recognize this track since we don't support this codec.

Quoting from https://www.matroska.org/technical/subtitles.html:

Storage of WebVTT in Matroska vs. WebM
Note: the storage of WebVTT in Matroska is not the same as the design document for storage of WebVTT in WebM. There are several reasons for this including but not limited to: the WebM document is old (from February 2012) and was based on an earlier draft of WebVTT and ignores several parts that were added to WebVTT later; WebM does still not support subtitles at all; the proposal suggests splitting the information across multiple tracks making demuxer’s and remuxer’s life very difficult.

I'm going to mark this issue as enhancement and assigning it to @icbaker who is the best person to look into this.