improved MediaPlayerElement

[touseefbsb]

Proposal: Improved Media Player Element

Summary

the default MediaPlayerElement control and its MediaTransportControls have a lot of room for improvement. Some of them are listed below.

Rationale

it will provide a head start for any media related application.

Functional Requirements

1. Closed caption button : By default the closed caption button is collapsed and only becomes visible when subtitles are available for the media, this works good with the embeded subs but not for external subtitle files, the button should be visible all the times when an additional button at the bottom to select an external subtitle file, this will open "FileOpenPicker" to select a single subtitle file, and filter will be limited to subtitle files supported by uwp apis. This is helpful because even when creating custom transport control we cannot simply edit it Flyout in the xaml because there is no flyout for it there, its probably added in code behind within the control.
   [ryandemopoulos] This is a good suggestion. I think we'd want to have more discussion about the right way to expose this functionality; I agree that the lack of support for external subtitle files is something we could improve upon though. When I read over the issues in this list, this one seems lower priority (rarer of an issue) to me compared to most of the other issues in this list--do you agree?

2. Codec Problem : This is probably the most long lasting problem of uwp media platform that it can only play a limited number of audio and video codecs, and developers have to use third party libraries to cope up with that which can create extra complications. there has been even a uservoice for this issue for some time but has not been fixed.
   [ryandemopoulos] I've passed along this feedback directly to the members of our Media team that handle codecs.

3. Pointer disappear : I think there can be a boolean property to enable this feature on the transport controls, the use case is pretty self explanatory , when the user moves the pointer on the controls the pointer should appear, otherwise when pointer is static for a couple secs and media is playing it should disappear ( right when the controls disappear themselves ) and this behaviour should only be applied if the pointer is on the controls, for example if the pointer is outside the mediaplayerelement and media is played the pointer should remain visible to interact with other controls on the app window.
   [ryandemopoulos] I tried this out, and indeed I noticed that the pointer is not disappearing. I believe this is a bug--and one that I'd like to see fixed soon. For now, we'll continue to use this issue to track that problem, but I'm going to chat more with our devs to see if we can get this fixed before the other MPE issues listed here.

4. Pointer hover Thumbnails : This is another feature which could be controlled with a boolean property on the controls. when the user hovers the pointer on the seekbar, a tooltip should show the thumbnail of the media from that specific position, this can be pretty hard to implement on an app to app basis, so if it is built in will reduce a lot of friction for the developer. it also has a uservoice
   [ryandemopoulos] Yes, this is a good suggestion. I agree it's probably becoming almost industry-standard at this point. I'd like to put this Issue (all of these suggestions that relate to MPE) in our Future backlog, and I suspect that this specific item would be one of the more compelling ones to fix whenever we free up resources to get to it.

5. Fluent Design : Some fluent design can be further added to the controls for example : acrylic background to the controls, themeShadow to the controls as well.
   [ryandemopoulos] Could you elaborate a bit more on what you'd like to see here? We already have Reveal lighting on the buttons and translucency on the buttons too. Is the main thing shadows--such as raised buttons or something of that nature? I'd like to know if you have specific design ideas here...

6. Command Bar of controls : the default style of media transport controls has a strange behaviour where the more button is collapsed by default, and even if somehow we manage to make it visible from the style and add more AppBarButtons to the command bar they should automatically go to the overflow flyout of more button ( when DynamicOverFlowEnabled=true ) but this behaviour doesnt work either.
   [ryandemopoulos] I'm not sure I fully understand the scenario. Could you give me an example to illustrate? Are you trying to see buttons specifically to the overflow area--if so, what buttons are you trying to add and where are they showing up?

7. DVD-BLURay Playback : dvd-Bluray playback is most requested feature from users for almost all video players, will add a lot of value on pc as well as on xbox as well.
   [ryandemopoulos] I've passed along this feedback directly to the members of our Media team that requests for DVD-BLURay Playback. Thanks.

if the team decides to go ahead with this, then I can suggest some more improvements or features as well, also maybe the community has some more ideas as well. I know some of these features can be implemented on app to app basis, but it would reduce friction to have them built in and reduce complications for the developer and they will be turned off by default and developer can just turn them on with a boolean property if they need the specific feature.

[ryandemopoulos]

@touseefbsb, thanks for the detailed write-up. I'm going to look into these points a bit more this week, but for full transparency I'm heading out for the holidays soon so it might get put on ice until the new year. I need to look into a couple things you wrote--I thought the Pointer Disappear logic was already implemented/working (so not sure if you've just hit a bug...?), and some of the suggestions are also things that I need to re-direct to other teams inside of Microsoft. (MediaPlayerElement doesn't actually own or provide any codecs; I'll have to re-direct that part to our internal Media team)

If you don't hear back until Jan 2nd, then happy holidays and chat with you more in the new year!

[touseefbsb]

@ryandemopoulos no problem, enjoy your holidays and after that we can continue the discussion.

[touseefbsb]

updated first comment with 2 more feature requests.

[touseefbsb]

just wanted to ping you @ryandemopoulos about this. I hope u had great holidays :)

[ryandemopoulos]

@touseefbsb yep, this is on my radar--might be another day or two more (early next week) before I respond as I'm crunching on a deadline. Happy New Year!

[ryandemopoulos]

@touseefbsb I've start a conversation with our Media team on this. Still following up; I hope to have more tomorrow.

[touseefbsb]

@ryandemopoulos thanks that is great news, will stay tuned in :)

[ryandemopoulos]

@touseefbsb OK! Thanks for waiting. After poking around on these issues, I figured the easiest way to provide an update here is to just put some comments into the Issue write-up itself, so check out the commentary I wrote in the Functional Requirements section.

At a high level, I think most of these issues probably belong on our Future backlog as things we can consider down the road, although I think we should fix the Pointer Disappear issue sooner since I'm pretty sure that's a bug and perhaps even a regression in functionality.

For the Codec/BluRay stuff, I have forwarded those on to the Media team for consideration since those aren't acutally controlled by MediaPlayerElement as I mentioned.

Again, check out my commments above directly in the write-up--I have a couple questions I put in there.

[touseefbsb]

@ryandemopoulos
ok here is the follow up on the questions in the comments.

Fluent Design : I have a couple ideas regarding it.

1. the ContolPanel is the section where all controls appear including slider, commandbar with its buttons etc, so maybe that whole section of xaml ( x:Name="ControlPanelGrid" is I guess its default name ) should cast a shadow on the media playing behind it, this will give a sense of elevation to the controls with respect to media, I know by default only the Top border of these controls are exposed and other 3 borders are directly connected with edge of mpe, but devs can easily change that even if they dont, then they will only see top shadow, which will also look very nice, just like many websites have a top navigation bar and their bottom border casts a shadow on the content below it, I think this is mostly common in material design.

2. in-app acrylic for the same ControlPanelGrid. ( I already have that in my app so I can help with that if required .

Command Bar

by default all media buttons are in the PrimaryCommands section of CommandBar, and when a normal command bar in uwp has less space to show them all, it automatically shows some buttons and puts rest of them on "overflow menu" and the 3 dotted "more button" also appears to access those buttons/controls in overflow menu. This is default behaviour of a commandbar considering "DynamicOverflowEnabled" property of commandbar is set to true.

In case of mpe controls it has a drop down order for controls to prioritize which will remain visible when less space is available, now the first problem with this behaviour is that it is not consistent, sometimes when mpe has less space it shows 3 buttons when it has space to show about 5 or so buttons. The second and more concerning problem is that those buttons who drop out in less space never appear in the overflow menu and the more button never appears visible to access the overflow menu.
Now I tried multiple workarounds and in some cases I forcefully make the more button visible and put some controls in SecondaryCommands to show them there, but I have not been able to get the dynamicOverFlow working as it should.

The ideal case would be that the buttons drop out as the dropdownorder has been set on them and all droped out buttons appear on the overflow menu and the more button automatically becomes visible when 1 or more items are in it, otherwise remain collapsed, and also controls should show as much buttons as possible on the space they have at any moment.

Closed caption : and yes we can discuss this more, but a typical good implementation can be seen in Movies and TV App. it shows a single button for audio tracks and subs and shows 2 lists in the Flyout of that buttons one for audio tracks one for subs which have listed internal tracks and internal subs respectively, then there is a button at the bottom which says "Choose subtitle file" and upon choosing an external valid subtitle file, that new entry is added to the list of subs available and is also selected automatically. This is a good implementation, one thing I would like to add to it would be a "Choose external audio track" button as well. And this also somewhat relates to codecs, bcz I am not sure if uwp supports all types of subtitle formats as compared to ffmpeg, so if the media team manages to support codecs they will probably also be able to support all types of subtitle formats.

Currently this is most active library for codecs, but of-course it would be much helpful if Microsoft resources are used to help move it forward or even best to integrate it into the platform SDK/winUI.
(https://github.com/lukasf/FFmpegInterop)

and regarding other features, I hope they are also tracked on this issue :) Thanks.

[ryandemopoulos]

@touseefbsb Thanks, appreciate these details. We'll keep this issue active in the repo and if/when we get time to focus more on improving MediaPlayerElement, we can revisit. I'm still planning to push the Pointer Disappear item now (vs. waiting until later) since that seems like it would be a super annoying bug. :) Glad you raised that one too.

[touseefbsb]

@ryandemopoulos Thanks that would be great.

[touseefbsb]

@jevansaks so does it mean work on it is going to start soon ?

[jevansaks]

No sorry, this was just cleaning up some project tracking metadata.

[ryandemopoulos]

Unassigning myself for now. Would like to revisit this down the road, but crunching on other items (including WinUI 3-oriented work) at the moment.

[hupo376787]

I have one more suggestion: Second subtitle.
Which means the player can show two subtiles, just as the picture shows.