Closed caption support in video_player

[localpcguy]

The video_player plugin does not seem to support Closed Captions. It is generally something that a video plugin should support for accessibility reasons. It is also a requirement for the app we are currently building in Flutter.

• Is support for closed captions something that is on the near (next month or two) roadmap for the video_player plugin?
• If not, would a PR adding that be considered for merging?

[bach942]

It would be great to have this feature. We've done both the CC in both ExoPlayer and AVPlayer at work, so it would be great to have a flutter plugin that supports CC and not have to write a platform wrapper that would do it.

[cbenhagen]

I think this might be too high-level for the VideoPlayer plugin as you would also need some controls for it and a way to style or position the captions. It might be better to add this to a higher level plugin like Chewie. There is on open issue for this here.

[localpcguy]

I disagree, accessibility would be better to have implemented as a core feature. Not everyone is going to be able to implement something like Chewie (i.e. if they need a custom UI) and if accessibility is something that Flutter is going to prioritize (as it should), at the least it should support the functionality.

The UI (mainly the CC button and where that lives) could be up to the implementer (and Chewie could provide a default), but the VideoPlayer plugin should support the necessary functionality that the UI would then utilize. I think the captions themselves do belong in VideoPlayer as core functionality, but how they are styled is something that the implementer of the UI could control.

[cbenhagen]

What about a completely separate CC plugin which can then be used by Chewie or other players? It would not care about drawing, but just read the subtitles files and provide a Listenable with the text to be rendered? A CcController which you then could pass to Chewie?

Adding this external to the core plugins would also allow us to move much faster with this while reducing the workload of the core devs.

[localpcguy]

If it isn't built into VideoPlayer, it puts the onus of accessibility on the individual teams to implement. If Flutter's goal is to be an accessible platform by default, then I still maintain it is core functionality.

That said, a separate plugin would be better than relying on a specific UI plugin implementing it. But I don't personally believe it's ideal from an accessibility perspective.

[bach942]

I would agree with @localpcguy is saying. That since flutter is pushing accessibility, it should be built into the core video player and not an add on plugin. It may speed up the development of the core for the video player development, but at the expense of the implementing developers.

[localpcguy]

I also wanted to add that if it isn't in core, it is, IMO, less likely to be implemented. Even if there is a plugin available, it's just another friction point and will lead to lower accessibility in videos implemented with the VideoPlayer plugin.

[msal4]

Is there any update on this one?

[snaeji]

It's in the p2 goals so hopefully soon....

[vanlooverenkoen]

I would like to see it in the core package as well. Any update?

[rakyat-game]

Minimal update only support SubRip(.srt) type

Update to core plugins video_player two files:

• android/src/main/java/io/flutter/plugins/videoplayer/VideoPlayerPlugin.java
• lib/video_player.dart

For displaying please look at
This

File Attachment
VideoPlayerPlugin.zip
video_player.zip

[adar2378]

@rakyat-game you should ask them to merge your code if it is stable

[localpcguy]

As part of this story I think it should handle having the close captions as part of the video stream, not just as a separate SRT file.

Also, @rakyat-game is there a PR that is available to pull down to test?

[localpcguy]

I agree, some support is better than none! I should have clarified that I'm excited to see progress on this issue. I was just noting that so either this issue can stay open (or if this is closed a new issue opened) to track embedded close caption support as well.

[snaeji]

I was able to build on @rakyat-game code and read VTT files and default to subtitles in the HLS stream.

The solution is rather dirty, language default is hardcoded and defaults to Icelandic subtitles if they are included in the HLS stream.
But this will maybe help somebody if he wants to use it for reference etc.

https://github.com/snaeji/video_player-0.10.1-3
https://github.com/snaeji/chewie-0.9.7

Update:
video_player doesn't default to icelandic anymore.
video_player fetches all subtitles from HLS and chewie gives an interface for selection of subtitles.
** Android only

[Frank-CG]

@snaeji , thanks for the solution of https://github.com/snaeji/video_player-0.10.1-3, it works fine for Android, but it seems doesn't support for iOS platform, is there any solution for this?

[snaeji]

Hey @Frank-CG , I'm sorry that i didn't mention it but this solution is for Android only. I know that you can sideload subtitles on IOS and of course display subtitles from hls but not sure how you would fetch the data and send it over the video_player plugin.

And you're welcome!

[Frank-CG]

@snaeji , thanks for your recommendation, I want to display closed caption that embedded in hls streams, I'm trying to work on it.

[snaeji]

Good luck! Wish i had more knowledge on the iOS side but no help from me.

[Joran-Dob]

Is there an update on this? This is a critical feature of our app.

[localpcguy]

@snaeji We have implemented CCs before natively in both iOS and Android and could potentially provide support in the form of answering questions or pointing in the right direction for iOS. We've been monitoring this issue and may even try to contribute a solution once we can find the time (working at a startup, so super busy right now just getting the basics done).

[Joran-Dob]

I now created my own package for this implementation in the dart, it wraps a video player and accepts a video player in its controller to now the current position.

You can find it here, hope it helps

[rlee1990]

Any updates this should be set to a critical level without CC a big amount of customers are left out.

[johnsonmh]

I'm going to look into this a little today. Can't promise anything, but it might be interesting to have...

• Another package for parsing common closed caption file types into dart objects.
• The video_player package could depend on that package and add support for associating a parsed closed caption file to the VideoPlayerController.
• VideoPlayerController could have a field caption, that gives the caption for the current position (or null if there is no caption for the current timestamp).
• video_player could have a basic widget for displaying CC that would take in a VideoPlayerController, and read the caption field. Although there would be no requirement that users use this Widget because other libraries could display CC however they want by reading VideoPlayerController.caption.

[abdoutech93]

This #2473 Pull Request support hls, I mean if this closed caption will be get from hls file ? When that will be merged ?

[rlee1990]

Any update on when this will be released?

[johnsonmh]

flutter/plugins#2487 has landed and version 0.10.7 is live on pub.dev! These changes include support for supplying a ClosedCaptionFile to the VideoPlayerController, and reading VideoPlayerValue.caption in your app. See the example app for a basic example of how to use closed captions in your app.

Right now the only file type that is supported out of the box is SubRip, but anyone can implement and use ClosedCaptionFile with VideoPlayerController!

I'm going to close this issue now that basic support exists, any future bugs or feature requests (such as supporting additional file types) should be filed separately.

[johnsonmh]

@JohnGalt1717 Happy to look at supporting file types that people would find useful. Please file your feature request (or pr!) separately. You can reference this issue in subsequent issues or pull requests.

[snaeji]

I agree with @JohnGalt1717 that we should not close this issue since it's a major accessibility concern. When live streaming HLS for example the subtitles are embedded in the stream. Therefore we have to access them on the video_player level.

[johnsonmh]

Thanks for the feedback, I've filed #50595 to track this issue, please add as much detail as possible (and add thumbs up) to that issue. Specifically, it would be helpful if you could give an example of a HLS that demonstrates what type use case you want to have supported.

[github-actions]

This thread has been automatically locked since there has not been any recent activity after it was closed. If you are still experiencing a similar issue, please open a new bug, including the output of flutter doctor -v and a minimal reproduction of the issue.