Add quality selector for progressive download videos

[LukasKalbertodt]

After much discussion, some relevant changes have been merged into paella-core and the other part is published as this package. So we can use this modified video format plugin to get back mp4 quality selection.

I have tested it a bunch and so far everything works fine. But please everyone: test this a bunch on different devices and different browsers and report any problems or weird behaviors here. Ideally test a number of different videos (single & dual stream). Note: The "The best open cat videos" do not have different qualities, so you can ignore those.

Test here

There are two known shortcomings still:

• When changing quality, both videos are loaded sequentially, which is slower in most cases than loading them at the same time. To fix this, changes in paella-core are necessary. These might be added in the future.
• There is no spinner or any UI at all indicating to the user that the new quality video is being loaded. The playback stops and potentially the videos are going black and then showing a frame again, but this is not ideal from a UX perspective. However, having the quality changing feature at all was the priority, and this might be added in the future still.

This pull request also changes the logic for selecting the initial quality. Before, the maximum quality was always chosen, which is really not ideal most of the time. You can find some additional information in this commit description. Please also test this change and let me know if you encounter a situation where the initial quality was a bad choice. Also note: the new logic still errs on the side of "high quality". I'm sure for many institutions it might be useful to lower the initial quality even more.

[LukasKalbertodt]

Things I tested:

• Changing quality while paused: should remain paused until manually resumed.
• Changing quality while playing: should pause until the new quality is loaded and then automatically resume.
• During quality change, it shouldn't show/flash the first frame of the video. Showing black briefly is not ideal but fine.
• All these things shouldn't change after changing the quality: playback rate (speed), volume, ...

But try to come up with test conditions for yourself, I likely forgot something here!

I tested:

• Linux; 4k screen; Firefox and Chrome: works (initial quality: 2160p)
• Windows 10; 1440p screen; Firefox, Chrome and Edge: works (initial quality: 1080p, 1440p wasn't availabe in the videos I tested). Firefox sometimes flashed a wrong frame during quality change, but I couldn't properly reproduce that :/ Shouldn't block merging this I think.
• iPhone 13 Mini (Safari): works (initial quality: 720p). For one video, a slightly zoomed in version of the video is shown for a split second after the quality change. I'm not exactly sure what I'm seeing. Also shouldn't block merging this I think.

I also noticed that the new fake fullscreen mode (used on the iPhone) makes it impossible to see Paella menus (like the quality or speed). This has nothing to do with this PR, but I will try to fix it. Fixed

[owi92]

I did some testing as well with Firefox, Chrome, Safari and my ageing android phone.
I noticed one thing that I can only reproduce on this test deployment:

• Some videos may start while the loading icon is still shown. This happens for example with https://pr1143.tobira.opencast.org/!v/JcU4iuqj4LT and https://pr1143.tobira.opencast.org/v/HO499MHSQuR
  As far as I can tell this only happens with dual stream videos, though I didn't test huge variety. Can you check whether you can reproduce this?

Other than that, my testing suggests that this seems to work as intended.

[LukasKalbertodt]

Some videos may start while the loading icon is still shown. This happens for example with https://pr1143.tobira.opencast.org/!v/JcU4iuqj4LT and https://pr1143.tobira.opencast.org/v/HO499MHSQuR
As far as I can tell this only happens with dual stream videos, though I didn't test huge variety. Can you check whether you can reproduce this?

I can. Dang, I saw this before but assumed this was already the case before. Will investigate next week.

[LukasKalbertodt]

Okay, this is not caused by my changes to the quality logic, but by another unrelated change, that landed between 1.46.6 and 1.47.1. And since this PR updates paella-core, the bug is included. I created an upstream PR: polimediaupv/paella-core#359

Once that is merged and released, I will update this PR.

[snoesberger]

I did test this on my MacBook Pro with Safari, Chrome, Firefox and Opera. The testcases @LukasKalbertodt mentioned did all work well. But I do have one small issue with the video https://pr1143.tobira.opencast.org/v/AYrn0crKqY1. If I switch the quality in Firefox, for a short moment the first video frame (or is it the preview image?) is shown. This happens only for this video and only in Firefox on my MacBook. If I open this video on my iPhone 12 Mini in Firefox, it is working as expected.

On my iPhone 12 Mini the tests were also successful both with Safari and Firefox.

[LukasKalbertodt]

@snoesberger Thanks for the tests. I can reproduce the Firefox issue. I started working on a fix and I think I have an interesting approach (for devs: creating a new video element via oldVideoElement.cloneNode(), setting the new src, waiting for "canplay" event, then oldVideoElement.replaceWith(newVideoElement)). This might be better in many ways anyway, but it requires more work and is also finicky to get right (cloneNode() does not clone event handlers, nor volume, and potentially other things). So in the interest of time and priorities of tasks, I would suggest not addressing this issue for now. Is that fine by you?

[snoesberger]

So in the interest of time and priorities of tasks, I would suggest not addressing this issue for now. Is that fine by you?

That is fine for me, we can leave it like that for the moment.

[LukasKalbertodt]

With 1.48.1, the problem that @owi92 found seems to be resolved. If you can confirm that Ole, I think this is ready to be merged.

[owi92]

With 1.48.1, the problem that @owi92 found seems to be resolved.

Can confirm.

[oas777]

Late to the game and moving on thin ice: In the old ETH video portal, we don't offer quality selection in "embed mode", because the space available for the video has less pixels available than the higher resolution one might chose. And we don't expect anyone to chose lower resolutions in a scenario like https://video.test.tobira.ethz.ch/!v/AUGZS5hMGIy. So maybe it makes sense to restrict the quality selector for fullscreen scenarios?

[oas777]

PS:

Why?

[LukasKalbertodt]

Late to the game and moving on thin ice: In the old ETH video portal, we don't offer quality selection in "embed mode", because the space available for the video has less pixels available than the higher resolution one might chose. And we don't expect anyone to chose lower resolutions in a scenario like https://video.test.tobira.ethz.ch/!v/AUGZS5hMGIy. So maybe it makes sense to restrict the quality selector for fullscreen scenarios?

I assume by "embed mode" here you just mean "not fullscreen mode"? I don't quite yet understand the reasoning for hiding the quality selector there, but in any case: in Tobira, the player is much larger in "non-fullscreen mode", so is much closer to fullscreen and thus I think that a quality selector certainly makes sense there too.

Why?

Are you referring to the unusual resolutions?

[oas777]

I assume by "embed mode" here you just mean "not fullscreen mode"? I don't quite yet understand the reasoning for hiding the quality selector there, but in any case: in Tobira, the player is much larger in "non-fullscreen mode", so is much closer to fullscreen and thus I think that a quality selector certainly makes sense there too.

The reasoning was/is that we provide more pixels (here: 1920x1080) than the regular monitor will allow you to use in that space. But maybe that's me staring at my laptop monitor all the time.

And yes, 576 seems an odd resolution.

[LukasKalbertodt]

The reasoning was/is that we provide more pixels (here: 1920x1080) than the regular monitor will allow you to use in that space. But maybe that's me staring at my laptop monitor all the time.

Ah I see. But as I said: I think it's quite different in Tobira, as the non-fullscreen player is already larger. I think conditionally hiding the quality selection will only cause more confusion.

And yes, 576 seems an odd resolution.

This is actually one of the standard vertical-resolution of DVDs! https://en.wikipedia.org/wiki/DVD-Video
But yeah anyway, not a resolution one certainly ones on your video portal. But luckily (for me :D) the provided resolutions come from Opencast, and Tobira cannot do anything about it. So yeah, if your Opencast provides nicer resolutions, then this is not an issue for you.