High-level design document

[chrisfillmore]

My team is going to look at shaka-player-embedded to try to get a sense of its capabilities. Currently we deliver video playback on iOS through Apple's native AVPlayer.

Something I haven't really seen in the docs (sorry if I've missed something) is a high-level overview of how shaka-player-embedded works. Are you bundling Chromium? Where does the video playback surface come from (i.e. the "video element")? Are there any known limitations compared to using Apple's native tools?

Any information is great. Thanks so much.

[theodab]

Well, compared to an AVPlayer, you won't have the same native AirPlay support. Because most of that stuff was hidden behind private APIs last I looked into it.

[TheModMaker]

We don't bundle Chromium, especially since I think it doesn't do video rendering on iOS. I think Chrome just uses the native web views since iOS has restrictions on HTML rendering. We use JavaScriptCore directly as the JavaScript engine and implement any of the required web APIs.

For video/audio playback, we use FFmpeg to demux/decode the content (using native iOS hardware acceleration) and draw the texture onto an AVImage. The C++ API returns the frame to the app to draw it as they see fit (see the demo). If you use the Objective-C/Swift API, it draws the image on the ShakaPlayerView. So the ShakaPlayerView (which is a UIView) acts as the video element. Audio is played using SDL on the default audio device (see #31).

We don't have AirPlay or FairPlay support and by default we only support H.264+AAC in MP4 (although you can change this with command-line flags). We also don't support direct playback of media files, we only support HLS or DASH.

But unlike AVPlayer, we also support playback of DASH content and clear-key encrypted content.

We should add a high-level design document.

[chrisfillmore]

Thanks for that info! We are definitely interested in DASH-WV support on iOS. I may come back with more questions if my team has some.