New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Media player support #196

Closed
matthidinger opened this Issue Apr 4, 2017 · 31 comments

Comments

Projects
None yet
8 participants
@matthidinger
Member

matthidinger commented Apr 4, 2017

Implementation status

Problem

It is not possible to embed media elements in an Adaptive Card today. Media would initially support audio and video.

Scenario Requirements

  • Authors need to be able to provide both video and audio. Some bots only send audio snippets.

  • Media does NOT auto play.

  • NEW: Media is NOT downloaded until user clicks play

    • That means in HTML, set preload="none"
  • Hosts can choose to allow inline playback (or disable interactivity completely as usual).

  • By default, without any host changes...

    • Inline playback is enabled
    • Videos/audio are fully functional, hosts don't need to do any work!
  • For video...

    • Authors can provide a thumbnail (poster) to be shown
    • If author didn't provide a poster, host's default poster is used (or blank/grey/black if host didn't provide a default poster)
    • If host disabled interactivity, stop here
    • Otherwise, a default play button should be overlayed and centered horizontally/vertically (hosts can override this play button)
    • Clicking the entire thumbnail+play button should...
      • If inline playback allowed...
        • Overlay play button disappears
        • Media controls appear (native OS media controls)
        • Video starts playing
      • Else if inline not allowed...
        • Simply raises an event to the host, host is then responsible for playing the video
  • For audio...

    • Authors can provide a thumbnail (poster) to be shown
    • If author didn't provide a poster, host's default poster is used (or blank/grey/black if host didn't provide a default poster)
    • If host disabled interactivity, stop here
    • Otherwise, a default play button should be overlayed and centered horizontally/vertically (hosts can override this play button)
    • Clicking the entire thumbnail+play button should...
      • If inline playback allowed...
        • Overlay play button disappears
        • Media controls appear (native OS media controls)
        • Displayed poster should remain displayed
        • Audio starts playing
      • Else if inline not allowed...
        • Simply raises an event to the host, host is then responsible for playing the audio
  • If hosts disabled inline playback, hosts need to know when the media element was tapped, so that they can handle playing the media themselves

  • Hosts need to be able to limit the max download size of the http media, so that a 1 GB file doesn't get downloaded, for example.

  • Authors can provide AltText for accessibility purposes

  • Authors can provide closed captions file

Asks

  • Cortana
  • Objects
  • Skype (in header, not body)

What Facebook does...

  • On Android...
    • When the video scrolls off screen, they pause the video
    • When user plays a different video, they pause the video
    • When user clicks already-playing video, it expands to full screen
  • On Desktop web
    • When the video scrolls off screen, they continue playing the video
    • When the user plays a different video, they continue playing the existing video too (both play at once)
    • User can click expand button to expand to full screen
    • They do pre-load videos, but they display thumbnail first

Cortana requirements

  • Events for...
    • Video start/stop
    • user pauses/stops/starts
    • user clicks away
    • video duration/elapse time
    • user skips ahead/back
    • user replays video
  • Ability to play/pause/skip video
  • Author option to auto-play

WebChat requirements

  • Controls for...
    • Entering full screen (would probably get user asks if that's not there)
    • Mute or volume control (probably only need mute)
    • Seek bar: At minimum -10s and +30s, depends on length, definitely in full screen people expect a way to seek
    • Captions (for accessibility)

Comps

Default state

The state when the card first appears. This is the state that appears regardless of inline playback allowed. If interactivity isn't allowed, the play icon shouldn't be rendered.

image

Inline playing/paused state

If inline playback allowed, clicking from the default state results in what's seen below: Play button disappears, poster disappears, and native OS media player and controls should be used, no specific requirements on buttons available.

image

Open Questions

  • Does the play/pause icon need to be adjustable?
  • Auto Play. Decision: no
  • We need to raise an event when the media is initially played from the poster preview, should we use the existing OnAction event or create a new one for OnMediaStarted?
  • Captions?
    • Decision: Make this part of 1.1... need to add that to spec schema
  • Max download size: what if someone attaches a 100MB mp4? Do we need to revisit host config max download sizes? Decision: Yes, we've added a media max size to host config which renderers must respect
  • Does the media element need to be declared in the header? Do hosts only want the video at the top of the card, and not in the free-form body?
    • We already know that Skype only wants it at the top
    • Andrew to reach out to Cortana/WebChat (7/30)
  • Cortana needs optional auto play (author specified)
    • Decision: Pushed to 1.2
  • Allow hosts to specify different default audio poster
    • Decision: Pushed to 1.2

Example Comps

image

Spec

Schema Changes

Media

Property Type Required Description
type "Media" true Must be "Media"
sources MediaSource[] true The media sources by MIME type. The first recognized format will be used. It is not supported to mix video and audio in the same Media element.
poster string false Strongly recommended thumbnail preview image for media. If not provided, host's default poster will be used (or blank screen shown).
altText string false Optional text to include information about the preview image for users who are visually impaired

MediaSource

Property Type Required Description
mimeType string true The MIME type of the media. E.g., "video/mp4". The primary portion must be either "video" or "audio"
url string true The URL of the media source

Example

{
  "type": "Media",
  "poster": "http://...",
  "sources": [
   { "mimeType": "video/mp4", "url": "http://.mp4" },
   { "mimeType": "...", "url": ".ogg" }
  ]
},
{
  "type": "Media",
  "poster": "http://...",
  "sources": [
   { "mimeType": "audio/mpeg", "url": ".ogg" },
   { "mimeType": "...", "url": ".ogg" }
  ]
}

Host Config

The following options allow hosts to override the defaultPoster if the card does not include a poster. It also allows customization of the play overlay button before the media has played.

{
  "media": {
      "defaultPoster": "http://...",
      "playButton": "http://...",
      "allowInlinePlayback": true|false,
      "maxMediaSize": 1024 // 1 MB
   }
}

Media states

Each platform represents media states in different ways. Listing them out here to determine the best union for what we want to use.

UWP Android iOS (AVPlayerItem)
MediaOpened setOnPreparedListener() (media loaded and ready) AVPlayerItem.Status.readyToPlay
Buffering MEDIA_INFO_BUFFERING_START, MEDIA_INFO_BUFFERING_END AVPlayerItemPlaybackStalled
Closed (no media)
Opening
Paused Override pause() in MediaController AVPlayerTimeControlStatusPaused
Playing MEDIA_INFO_VIDEO_RENDERING_START AVPlayerTimeControlStatusPlaying
Stopped setOnCompletionListener() (end of media reached) AVPlayerItemDidPlayToEndTimeNotification

Renderer Requirements

  • A Renderer MUST enumerate the list of sources and use the first recognizable mimeType to determine the type of media (audio or video)
  • A renderer MUST display video/audio as described in the scenario requirements in For video... and For audio...
  • A renderer MUST raise OnMediaClicked event (using whatever local naming schema makes sense) ONLY if host disabled inline playback, so that host can display the media
  • A renderer MUST respect the maxMediaSize from host config to prevent accidental download of excessively large files.
  • A renderer MUST provide play/pause controls. It SHOULD include volume controls if possible. Using native OS media controls is fine.
  • If sources contains an invalid mix of MIME types (audio and video) the element MUST be dropped and a warning SHOULD be logged.
  • A renderer MUST support and follow http redirects in the URL source

Media features intentionally left out

The following options are not provided to card authors.

"controls": true, // implicit true
"loop": true, // no auto loop
"autoPlay": true, // no auto play

Reference

For reference, HTML supports these:

  • MP4 = MPEG 4 files with H264 video codec and AAC audio codec
  • WebM = WebM files with VP8 video codec and Vorbis audio codec
  • Ogg = Ogg files with Theora video codec and Vorbis audio codec

Downlevel Impact

High

Preview image will display, but any actionable media will not play. It is important to not display play button when on a 1.0 renderer to limit confusion.

1.2 features

  • Optional auto-play (hosts have supreme authority, authors can choose whether their video should auto-play)
    • This is needed by Cortana, when you ask "Show me my home security camera", you want that to auto-play, but when you say "Videos of cats", you don't want those to auto-play
  • Host ability to provide a different default poster for audio (defaultAudioPoster?)
  • Host APIs to access native media element, so they can access all events (like play/pause/seek), and so that they can programmatically play/pause
  • Potentially future scenario: Host events for OnMediaStarting, OnMediaPaused and OnMediaStopped events to allow hosts to know the state of the media
    • This could instead be solved by accessing native media element?
  • Hosts need to know whether the media is currently playing, so that for scenarios like notifications, they know to keep the notification on-screen rather than letting it time out. Therefore, they need to know when the media started playing (hold on screen). And they need to know when the media was paused (possibly release hold on screen?) and when the media ended playing (definitely release the hold on screen?).
    • Could use the two above features for that?
  • Host option to choose to enable pre-load (so that video plays instantly when clicked, but therefore uses more data)
@danmarshall

This comment has been minimized.

Collaborator

danmarshall commented May 26, 2017

Would really like this for WebChat.

@danmarshall

This comment has been minimized.

Collaborator

danmarshall commented May 26, 2017

Also, there may be Audio player support.

@mdtauk

This comment has been minimized.

mdtauk commented May 29, 2017

It is useful to include this, but give it some careful thought. As a user, I do not want video notifications or live tiles appearing on my PC or Phone. Consider implementation rules for auto-play, volumes etc

@matthidinger matthidinger added this to the v1.1 milestone Aug 22, 2017

@khouzam khouzam removed this from the v1.1 milestone Nov 30, 2017

@matthidinger matthidinger changed the title from Video player support to Media player support Feb 8, 2018

@andrewleader

This comment has been minimized.

Collaborator

andrewleader commented May 16, 2018

From Action Center developer platform side (uses UWP renderer)...

  • Missing renderer requirement that downloading the resource must go through the ResourceResolver, so that we can download the resource ourselves through our own caching code
  • We would need the OnMediaStarted and OnMediaEnded event to know to keep the toast on screen while the user is watching it, and then 5 secs after the media ends, dismiss the toast (unless more user interaction is occurring)

We would need someone from the UX side to chime in on this, but I'd imagine we initially would disable the inline playback.

For hosts that don't support hit targets and interactivity (like Timeline), they would need a way to configure it so that it just displays the thumbnail (without any play button), since it's only the entire card that is clickable, rather than specific content on the card that's clickable. Seems like that might already be possible with what's described here but writing that down just to make sure.

@dclaux

This comment has been minimized.

Member

dclaux commented May 24, 2018

The spec lacks size handling guidance for video. I think it should work like this:

  • Width is always 100% of the parent container's
  • Height is dynamically computed by the media player so as to respect the video's aspect ration

What's more problematic IMO though (and I'm talking about the HTML implementation but it would probably apply to all platforms) is that each browser has its own controls implementation; they all look very different and they can't be styled. They will look quite bad inside a card, especially the audio player which can't even be sized.

Therefore I believe we should own the controls, but that's a significantly higher investment unless we keep it simple, e.g. play/pause, maybe mute, but no seek nor volume sliders.

@dclaux

This comment has been minimized.

Member

dclaux commented Jul 13, 2018

Doing the play overlay button as an image in HostConfig is actually not that great a solution. Allowing the styling of the button to be done in CSS is a much better option and satisfies the customization needs. Besides, the design, at least for the HTML media player, calls for more than just the play button to be done as an overlay, and therefore additional properties would have to be added to HostConfig to handle the other button(s).

I suggest we either:

  • Change the design and remove the property from HostConfig, and let hosts customize buttons via other means
  • Or make honoring the HostConfig property optional if the host has a better way to handle customization.
@dclaux

This comment has been minimized.

Member

dclaux commented Jul 13, 2018

I'm just discovering that since we last discussed and agreed on the spec, new requirements have been added, namely OnMediaClicked, OnMediaStarting, OnMediaPaused, OnMediaStopped (what's the difference between puase and stop?) and maxMediaSize (not sure about that last one though).

These were never reviewed. What gives?

@andrewleader

This comment has been minimized.

Collaborator

andrewleader commented Jul 13, 2018

The original spec was ambiguous and missing elements. For example, it didn't distinguish whether OnMediaStopped included paused (which would be needed by hosts), and it was completely missing scenario requirements.

In fact, we still don't have full scenario requirements. I'm trying to talk to hosts today to determine the actual scenario requirements so that we can be sure we have the correct set of events (hosts might even need to be able to dynamically pause the video, for example)

@dclaux

This comment has been minimized.

Member

dclaux commented Jul 13, 2018

There was no ambiguity: these events were not part of the original spec, and we never reviewed them. Plus, these events don't need to be spec'ed here because they have nothing to do with the Adaptive Card format, they are features of the renderer that can technically be exposed in different ways depending on the platform.

The spec above says "potentially, hosts need to be able to..." which doesn't seem like a strong requirement nor that there is a concrete scenario that needs the feature.

@dclaux

This comment has been minimized.

Member

dclaux commented Jul 13, 2018

As a matter of fact, you added all these to the spec 2 days ago @andrewleader.

@andrewleader

This comment has been minimized.

Collaborator

andrewleader commented Jul 13, 2018

Yes there was ambiguity, that is an absolute fact witnessed by the devs who were attempting to develop this and had questions, since it was ambiguous (for example, the spec said that first frame of video is displayed unless author provides a poster, but the spec also said that the poster is displayed (or host default poster is displayed), so it's ambiguous what's supposed to be displayed.

The "Potentially" one is an incomplete requirement, that's why it says potentially. We're trying to gather scenario requirements from hosts to determine whether those are needed.

@andrewleader

This comment has been minimized.

Collaborator

andrewleader commented Jul 27, 2018

New open issue: What should the size of the play icon be?

The spec currently just says to overlay and center the play icon image (which hosts can provide). That seems to imply that the play icon should be rendered however big the source image is.

The problem with that is, what if a host wants to provide a high res play icon image? Say they provide a high-res 400x400 play icon… and the media element itself is only 500x500… 90% of the content would be covered by the play icon!

Seems like it’d make more sense if the play icon is always a fixed size. Say 64x64 or something. Then, regardless of the resolution of the incoming image, the results are always consistent.

Additionally, seems useful to let hosts override that fixed size. They might need more than 64x64, or they might want something smaller.

Thoughts? Should we make those changes? What default size should we go with? Should we allow hosts to pick their own size?

@dclaux

This comment has been minimized.

Member

dclaux commented Jul 27, 2018

My opinion, as I shared here is that there is no need to allow for Play button customization in HostConfig. This is going too far for very little value add. I think every host should simply be responsible for choosing their own defaults, including size etc.

Besides, the desired UI (for the HTML renderer at least) is this one:
image

So it would be one thing to allow for the Play button customization, but what about the Rewind button?

I think we need to change the spec. We can always add support for button customization later if we actually realize it's necessary, but it really doesn't strike me as necessary (nor practical) at the moment.

@andrewleader

This comment has been minimized.

Collaborator

andrewleader commented Jul 27, 2018

The play button customization is only for the initial play button... Once clicked, the actual play/pause/rewind/volume controls appear (which cannot be customized).

You can certainly imagine that Teams would want a purple-themed initial play button, etc. I'll check with some of the hosts to see if they actually require this functionality though, I'm not sure if they explicitly asked Matt for that functionality.

@dclaux

This comment has been minimized.

Member

dclaux commented Jul 27, 2018

The design for HTML calls for the same button to be used. The big play button changes into a big Pause button when clicked, then back to Play when clicked again. When the video is playing, the button automatically disappears if the mouse cursor moves outside the boundaries of the video or stays idle for a moment. It appears again as the mouse moves over the video.

I think it should be possible for a renderer, be it the HTML one or anything other, to achieve the above design without being tied to using two different buttons. Said another way, I don't think there should be a requirement as to the UI of the video player for as long as it exposes the functional features described in this spec.

@andrewleader

This comment has been minimized.

Collaborator

andrewleader commented Jul 27, 2018

The functional features in the spec require a initial play button to be displayed along with the poster, and when clicked, the poster disappears, video starts playing, and full media controls appear. If you disagree with the spec, then we have to have a conversation about that.

@dclaux

This comment has been minimized.

Member

dclaux commented Jul 27, 2018

What I do not agree with is that the initial Play button over the poster and the Play button once the video has started need to be different. I was hoping that my screenshot, explanation and further information here made that clear.

And if they do not need to be different, I do not see it as necessary to make the Play button customizable in HostConfig - especially given that the design for HTML also calls for a Rewind button that looks and feels just like the Play button.

So the bottom line IMO is:

  • Being forced into consuming the HostConfig customization for the initial Play button would hamper the ability of the HTML renderer to achieve what I believe is the optimal user experience
  • If we do keep that HostConfig customization, the HTML renderer would therefore not honor it
  • If it can be ignored (I should say if achieving the desired UI requires ignoring it), then its value is discussable

So yes, as I have already said, I think we should have a conversation around reworking the spec a bit.

@andrewleader

This comment has been minimized.

Collaborator

andrewleader commented Jul 27, 2018

Here's an example of why the initial play button and the media-loaded play button need to be different... in bots:

video in bots

Bots use media controls at the bottom, with a time scrub bar. But they want a large initial play button.

Facebook Messenger does the same...

image

image

In your case, where there's no media bar at the bottom, it certainly would make sense to re-use the same icon, but that's not the only case.

@dclaux

This comment has been minimized.

Member

dclaux commented Jul 27, 2018

So just to be clear - Bot clients that use the HTML renderer (WebChat for instance) will not get the experience you are showing above; they will get the experience I outlined and provided a screenshot of. In other words:

  • No tracking bar,
  • No volume control
  • No countdown
  • No full screen control

Just the two big play and pause buttons.

That is, as I mentioned before, unless:

  • Your team is willing to lend a hand and do the work
  • Or we rely on the default control layout provided by each browser's media player implementation, which is often ugly and isn't stylable

I do understand it's not the only case, but I am also saying that handling the richer case (like in the shots you included above) still doesn't require Play button customization in HostConfig, even if the buttons are different. The only value add of having it in HostConfig (with the complexities it leads to) is to, say, have the button as pictured in the bottom card of your first screen shot used in your second screenshot. It is not the case in your screenshot and the value add of supporting that is very, very low IMO.

@andrewleader

This comment has been minimized.

Collaborator

andrewleader commented Jul 27, 2018

Thanks, we'll have to check with WebChat if that's acceptable then. I'll start a mail.

@andrewleader

This comment has been minimized.

Collaborator

andrewleader commented Jul 27, 2018

Requirements from WebChat:

Full-screen: required, will probably get user asks if we don't have it.

Volume control: probably we will only need mute feature, but not fine-grained volume control.

Seek bar: need discussion. It depends on the video length. IMO, minimum bar is last 10s + next 30s. People expect a way to seek with full screen video.

BTW, we will need to support some forms of transcript for accessibility.

Can't believe we missed closed captions/accessibility! D'oh!

@dclaux

This comment has been minimized.

Member

dclaux commented Jul 28, 2018

Thanks @andrewleader. Now we need to have a conversation around it and decide together which of these requirements we decide to actually make part of the spec.

To be upfront, though, if we decide that any of these is indeed required, then the HTML implementation will simply use the built-in player on all platforms, enabling every feature that can possibly be enabled. The implementation will be super cheap, but there will be no way to re-style the controls.

However, even if HTML uses the built-in player, I still believe we should reconsider the spec for the Play button and even the Poster. Here is what I suggest and why:

  • Built-in players on most platforms handle the "initial state" (thumbnail + play button) automatically. It is disputable that the experience we'd provide with Poster + HostConfig-configured default Play button would be any better, and the spec doesn't explain why we want to do it that way in the first place. Here is the default state in Chrome, for instance (from https://www.w3schools.com/html/html5_video.asp) (note BTW that I believe the controls in Chrome were just recently redesigned, they didn't use to look that good AFAICR):
    image

  • Media player controls have events to indicate when the player couldn't load, allowing us to fall back to a degraded experience. I believe that's when the default poster should be used

  • In that degraded experience, at least in HTML, the default Play button is undesirable, because we already know the video can't be played and the best fallback experience we can implement would be to open it in another browser tab, which would use the same built-in player...

On the caption front, this is how it's done in HTML:

<video width="320" height="240" controls>
  <source src="forrest_gump.mp4" type="video/mp4">
  <source src="forrest_gump.ogg" type="video/ogg">
  <track src="subtitles_en.vtt" kind="subtitles" srclang="en" label="English">
  <track src="subtitles_no.vtt" kind="subtitles" srclang="no" label="Norwegian">
</video>

It is likely we will have to add another property to the Media element to support text tracks, although I do not know how other platforms do it.

While I am at it, I think we should also re-consider displaying the poster for Audio. Why can't card authors do a poster if they want to using an Image, and why is there a need for a default poster for an Audio stream?

So let's please spend some time soon and re-discuss this spec.

@andrewleader

This comment has been minimized.

Collaborator

andrewleader commented Jul 30, 2018

Reasons for the "initial state" (thumbnail + play button)

  • A default "thumbnail" provided by a video is never great. Anyone who's published videos on YouTube knows that - that's why thankfully they added the ability to provide custom thumbnails and also let you pick from three automatically grabbed thumbnails
  • By being given a thumbnail image, we don't have to download the video, saving unnecessary downloads. Otherwise, the media control would have to download the first 5-10 seconds of the video (the first frame of the video can often be blank, so first frame isn't an option). Users aren't always going to play the video, so downloading it can be unnecessary - we only want to download when it's actually requested by the user.

Although for HTML it looks like the HTML video player supports a poster property already? So is that what you meant by it's "handled automatically"? You meant we should just use that property?

Why can't card authors do a poster if they want to using an Image

They can? Isn't that what we wrote in the spec?

why is there a need for a default poster for an Audio stream?

The default poster is currently shared between video and audio... definitely might make sense to have those separated, so that a host could have none or a different default poster for audio.

@andrewleader

This comment has been minimized.

Collaborator

andrewleader commented Jul 30, 2018

I'll schedule a meeting with everyone soon!

@dclaux

This comment has been minimized.

Member

dclaux commented Jul 30, 2018

Many (most?) video files have an embedded cover image; retrieving it is hardly more expensive than downloading an image separately, and happens very fast.

I think we need to be realistic when it comes to the developer experience - do we really want card authors to generate custom, separate poster images for all the videos they put in their cards, even though the video itself likely already has an embedded cover picture? Do we expect that the Media element will be most used to display videos owned by the card author or videos owned by a third party? If the latter, do we really think it's a realistic/acceptable workfow for card authors to go extract poster from a video they don't own, and store it separately on their side?

You mention YouTube - our solution will not handle YouTube videos, since AFAIK one can only embed a YouTube player for that. Are we sure we do not want to support YouTube videos?

I meant Image with a capital I, as in an Adaptive Image element.

@dclaux

This comment has been minimized.

Member

dclaux commented Jul 30, 2018

More thoughts. Say we do require the poster as per the spec:

  • The Media element will stretch to the entire width of it parent container, in most cases the card itself
  • The aspect ratio of the video is a piece of information embedded in the video stream
  • The player in HTML (and most likely on every other platform) will adjust its height so as to respect the video’s aspect ratio

So we have a choice to make here:

  • We let the player pick its own optimal height in order to respect the video’s aspect ratio
    • In that case, what are the dimensions we recommend for the poster?
    • If the dimensions of the poster do not match the video’s aspect ration, the player’s height will change as the video starts playing, creating a jarring effect
  • We force a specific aspect ratio for the player itself, essentially we compute its height as a multiple of its width. Then we document that ratio as the one card authors should use for their poster. There are several drawbacks:
    • A widescreen video will likely always have horizontal black bars in the embedded player
    • A 3:2 video will appear smaller than it could with vertical black bars left and right
    • There is still no guarantee that the card author will have respected the documented recommended aspect ratio, so there is no guarantee the poster itself will fit well within the space reserved in the card, leading to either:
      • Black bars
      • Or the posted being truncated which could cut off the text it contains
  • We force the player into the poster's aspect ratio
    • Essentially we are in the same situation as the previous bullet, it's just that the size of the player has been determined differently (but still not based on the actual video's aspect ratio)
@andrewleader

This comment has been minimized.

Collaborator

andrewleader commented Jul 30, 2018

Good thoughts on the aspect ratio... Expanding on those some and giving them labels

  • Option A: Display poster and video each in their own aspect ratio

    • Means height might change when user first plays the video
  • Option B: Use the video's aspect ratio

    • Means we first have to pre-load some of the video (metadata) to obtain the aspect ratio (can be 44 MB as tested in HTML with Edge)
    • Not an option for some hosts due to size of download
  • Option C: Use the poster's aspect ratio

    • Means if the video is a different aspect, the video will have black bars
  • Option D: We always use a fixed aspect ratio, like 16:9

    • Means no matter what, some authors will always have black bars on their videos
    • Not a good option since lots of videos will suffer with no way of author fixing it

Seems like the best option is Option A. We would tell authors that they should use the same aspect ratio on the poster as the video, but worst case (should be uncommon), the height changes when it starts playing... but at least there's never any black bars.

The second best option seems like Option C. But I think the height changing would be preferred over black bars.

If we were allowed to pre-download some of the video, Option B would seem ideal. But even if we did that, the poster would get downloaded first, and then the video size info would come in later... so we still would have to make a decision on how to size the content before the video gets loaded (and we need to make a decision on how to size the content before the poster gets loaded too...).

Basically, there's three stages there...

  1. Nothing loaded yet - Use default 16:9 aspect ratio while requesting poster and video?
  2. Poster loaded - Now use poster aspect ratio?
  3. Video loaded - Now use video aspect ratio?

So the size could change twice within 2 seconds... that could be wonky. Then again, authors can fix this by just providing the correct aspect ratio.

So I say Option A is the best... it should be a minority that authors don't provide the correct aspect ratio, and even if they don't, a small jump in height won't be the end of the world.

@dclaux

This comment has been minimized.

Member

dclaux commented Jul 30, 2018

Looking at Edge and Chrome's default behavior, they both do A, e.g. they start with some specific height and then update it when they have loaded enough data to know what the video's aspect ratio is. So I agree A seems to be the best option here.

Given that it is also true we want to minimize to amount of data loaded (e.g. not load the video until the user decides to play it) and that my assumption that most videos have an embedded thumbnail image is apparently false (and even if it were true, browsers apparently all fetch the full video immediately as the page is loaded), I do agree that the poster is required.

I am still not sold on the HostConfig-configurable Play button though, but we can discuss that at the meeting Andrew will setup.

@paulcam206

This comment has been minimized.

Member

paulcam206 commented Sep 24, 2018

Documentation is done and in the mahiding/site1.1 branch.

@danmarshall

This comment has been minimized.

Collaborator

danmarshall commented Sep 24, 2018

cc: @compulim

@khouzam

This comment has been minimized.

Member

khouzam commented Sep 26, 2018

Completed for the 1.1 release

@khouzam khouzam closed this Sep 26, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment