HTML5 audio/video always automatically plays in QWebView on Windows

[dougmassay]

QtWebKit 5.212.0 Alpha 2 (precompiled binaries) with precompiled Qt5.9.1

When loading a local html file with html5 multimedia into QWebview, the video automatically starts playing -- regardless of any autoplay/preload attributes. If multiple html5 audio/video tags are on one page, they all start playing simultaneously as soon as the page loads. This doesn't happen with the same code on the Archlinux provided QtWebKit 5.212.0 Alpha 2.

Is there some QWebSetting attribute I'm simply overlooking that will remedy this on Windows, or is there an issue? The audio/video plays fine, it just won't wait on the user to click play.

I'm attaching a dead-simple project that exhibits the behavior. It consists of a pro file, one source file, and the html file (and mp4 video). The path to the html file to load is hard-coded -- sorry, I was in a hurry.

qwebview_browser.zip

EDITED TO ADD: my testing is on Windows 7 (both 32- and 64-bit).

[dougmassay]

I was, at the very least, hoping someone might tell me if my issue could be duplicated on other Windows machines (using my code snippet provided). Or perhaps acknowledgment that it is, in fact, an issue. But I did not expect crickets.

HTML5 multimedia should not be auto-playing without an autoplay attribute present in the html. So why does it do so using the precompiled 5.212.0 Alpha 2 binaries on Windows 7 when loading a local html file/mp4 into a QWebView object?

[annulen]

Could you check if it reproduces with Alpha 1?

[dougmassay]

Are there Alpha 1 binaries?

I could attempt to build Alpha 1 myself, but I'm not sure what options to use. Should I build Alpha 1 to use Media Foundation or QtMultimedia?

[annulen]

Yes, there are binaries in https://github.com/annulen/webkit/releases/tag/qtwebkit-5.212.0-alpha

Bug is probably specific to MediaFoundation, alpha 1 binaries didn't use it

[dougmassay]

I must be missing something obvious. I see no binaries for that release available for download. Only the 12Mb source tar.xz.

[annulen]

Oops. Use TP5 binaries

[dougmassay]

Will do. I'll report back.

[dougmassay]

Swapping out Alpha 2 for TP5 results in the video not loading or playing at all (using the same attached code snippet). Qt5.9.1 includes both the wmf and direct-show multi-media plugins, but I have no idea which one will be used.

[annulen]

It uses wmf and requires codecs to be installed

[dougmassay]

Hence my excitement to use Alpha 2 where 3rd-party codec installations are not required (for the codecs that WMF can handle inherently, anyway). As I said, the html5 video/audio elements play fine with Alpha2 (without any extra codecs being installed). The mp4 video I'm loading is h.264. Which WMF can handle inherently from Windows 7 on. They just autoplay with Alpha 2 instead of waiting for user input.

Are you saying I need to install codecs to play html5 h.264 video in QWebView (with TP 5) when WMF (which QtMultimedia is supposed to be using via a plugin) can play the video just fine without extra codecs?

[annulen]

WMF media player backend of QtMultimedia is disabled in official builds for a long time, IIRC since Qt 5.7 or 5.6. It's possible to use custom build of QtMultimedia configured to use WMF, @Emdek was doing this with satisfying results.

QWebView is supposed to play everything that QtMultimedia can play.

[dougmassay]

QWebView is supposed to play everything that QtMultimedia can play.

Well if the WMF backend is disabled in Qt5.9.1, then that explains why TP5's QWebView can't play the video (with no codecs installed). And Alpha 2 can play the video (with no codecs installed), but won't wait for user input to launch. Hell of a Catch-22. :)

I'm one tiny bug away (the autoplay problem) from being able to use precompiled binaries for both Qt and QtWebKit in my project, and not requiring users to download/install 3rd-party codec packs for mp4 video support. Damn.

Looks like I'll have to keep custom-compiling these monster dependencies just to be able to provide the possibility of hit or miss html5 video support in my project on Windows.

[dougmassay]

I'd love to hear if the compilation of my code snippet produce different results on Windows 8 or 10. But I'm not much of a Windows guy any more, so Windows 7 is what I have to build/test on.

[annulen]

@Emdek reproduced your issue on win 10

[annulen]

Issue is reproducing only with local files (generally speaking, it probably affects any medium where playback is possible right away without any buffering)

[dougmassay]

Thanks for the follow up. That's unfortunate since my open-source project is EPUB ebook editing software where media will nearly always be local. I suppose I can systematically enable/disable all QWebSetting attributes to see if some combination might magically accommodate local media, but I don't hold out a lot of hope.

Does @Emdek have any links/details on enabling the wmf backend in QtMultimedia (or will local media have the same problem with that route, too)?

[Emdek]

@dougmassay, no links, I've done it long ago.

[dougmassay]

No problem. Thanks anyway. I'll dig for it. :)

@annulen : mind if I keep this open 'til I can confirm that no combination of QWebSetting attributes will make local media behave properly (RE autoplay) with Alpha 2?

[annulen]

Well, it's a bug and its possible to fix it

[annulen]

@dougmassay When video is incorrectly autoplaying, what are value of paused and readyState attributes of <video> element?

[dougmassay]

readyState is 0 (HAVE_NOTHING)
and paused is "true"

Throughout the entire time the video is playing, and after it stops.

I'm just manually pumping out those values by clicking a button some javascript is attached to. Let me know if I should do it differently.

[annulen]

That's ok, or you can use Inspector to obtain them

[dougmassay]

I'll also note that the values are the same when manually replaying the video (after the initial incorrect autoplay is finished).

Correction: the paused attribute is "false" during the second--manual--replay and returns to "true" when it finishes playing. Sorry for any confusion.

The video cannot be replayed again after that (without reloading the page).

[annulen]

Probably we should finish preparation for playback on MESessionStarted, not on MEBufferingStopped. This might also fix bug with infinite loading, because current code checks that buffer is 100% filled after receiving MEBufferingStopped

[dougmassay]

If that's intended for me, I'm afraid it's a bit over my head. I'm not at all familiar with the codebase here (nor with MS's Media Foundation api).

[dougmassay]

Hacking around in MediaPlayerPrivateMediaFoundation.cpp I was able to play with MESessionStarted as mentioned and got something working for my local media clip sample. So it IS possible.

Unfortunately, my changes broke things for remote media--likely due to my unfamiliarity with the code. Hopefully someone more knowledgeable can get it accommodate both.

[dougmassay]

If I try to play with this a bit, should I be using qtwebkit-stable or qtwebkit-5.212 (or somewhere else) to submit pull requests? Provided I can actually come up with a viable solution, of course.

[annulen]

Actually, it should be contributed to WebKit upstream. If that's hard for you, I can handle this.

https://github.com/annulen/webkit/wiki/How-to-submit-patches-to-webkit.org

[dougmassay]

I'd certainly appreciate it if you could handle it. I may get it eventually, but it would take me a good while just to get the "lay of the land," so to speak.

[dougmassay]

Any word on this getting contributed upstream? At the very least, I'd love to be able to cherry-pick a submitted patch to be used with my own build.

[annulen]

Have you improved your patch so it doesn't break anything?

[dougmassay]

I'm not sure I'm qualified to determine if "it doesn't break anything," or not. It works with my limited testing (for both local and remote media) for my specific needs, but I'm not overly familiar with the code at all (plus I'm admittedly a bit of a hack when it comes to this sort of thing). But if it gets the ball rolling, I'll certainly share.

mediaplayer_patch.zip

[annulen]

I'm not sure I'm qualified to determine if "it doesn't break anything," or not

Just check if 1) youtube works; 2) http://camendesign.com/code/video_for_everybody/test.html works; 3) vimeo works

[annulen]

I'm not an expert on this code as well but on the surface change looks correct and shouldn't break anything

[dougmassay]

Just check if 1) youtube works; 2) http://camendesign.com/code/video_for_everybody/test.html works; 3) vimeo works

The autoplay attribute (or lack thereof) works as expected for me for all three of those remote sites, as well as my local media test with my patch (using my simple test project).

[annulen]

I've uploaded your patch to https://bugs.webkit.org/show_bug.cgi?id=178693 (same patch as you attached but C++14 construction is used)

[dougmassay]

Thanks a lot! I certainly appreciate all of your help with this issue.

[annulen]

Patch merged to trunk and to qtwebkit-5.212

[GainSury]

In Mac, this bug reproduces. this bug happens from tp5 to 5.212 alpha 3