Video playback issues / inconsistencies in reflow column/paginated mode, depends on codec and GPU

[rkwright]

From Maryam Arabi (UniSIM) maryam@unisim.edu.sg:

I’m facing an issue with the videos in my ePub course. None of them play on Chrome/Readium, unless they’re set on Continuous mode. They work fine in other readers and platforms and they all worked a few months ago after testing.

I tried to look for the solution in your Forum but unfortunately couldn’t find one.

Pls note everyone in my department has the same issue viewing the videos of these two courses. I’ve shared one of them with you through dropbox. We all use Windows 7. Below is the info requested:

OS: Windows 7
RAM: 8 Gig
Browser: Chrome
Version: 39.0.2171.99 m

[rkwright]

I have uploaded the sample EPUB to here:

https://readiumfoundation.box.com/shared/static/5gvl49qhtrf3mow53aqhij3jn0zb0xf5.epub

I have also reproduced problems on OSX with the current Chrome build. The first video in Chapter 1 works, but the second and third do not.

[danielweck]

UPDATE: I am not able to reproduce (OSX, Win)
This sounds like a regression bug:
(Windows-only, OSX continues to work fine)
#239
...yet again, browser and codec -specific video rendering bugs strike. What a pain!

[danielweck]

The last spine item of cc-shared-culture-20120130.epub works fine, yet the video is not in the first CSS column...now checking with the linked 132MB EPUB (Chrome, OSX).

[danielweck]

Ah, it's a Windows bug (cannot reproduce in OSX).

[danielweck]

@rkwright Ah, I cannot even reproduce the bug in Windows 8.1 (latest Chrome version).
In fact, if I remove the code added here:
readium/readium-shared-js@dc921ee
#239
...then the video doesn't show at all! (blank rectangle, no controls)
So I doubt that this is a regression bug on Readium's side. Maybe because of a Chrome update?

[rkwright]

I can reproduce it on OSX without difficulty. In fact, with the latest dev build on OSX (Yosemite) the first video doesn't play either. No poster, just the controls - which don't work.

Chrome: Version 39.0.2171.99 (64-bit)

[danielweck]

Works here! ??
Yosemite OSX
Chrome: Version 39.0.2171.99 (64-bit)
Tested with extension and cloud-reader (local Node-Express HTTP server)
At Chapter 1 "Welcome" of the linked 138MB EPUB, text under video is: "Welcome to the course ELT381e Singaporean Literature, a 5-credit unit (CU) course"
Latest code from develop branch, just rebuilt from scratch (grunt clean + grunt update-readium + grunt chromeApp / grunt cloudReader).

[rkwright]

Hm. Only difference I see (aside from my being in the Midwest ;-) is that I just created a new sandbox and did

grunt chromeApp

Let me re-test.

[rkwright]

Same result. My versions are:
2.16.0
Wed Jan 21 2015 09:14:06 GMT-0600 (CST)
readium-js-viewer@b7d480bc10c9faaca2a5dbf3fa7af2a52a9541cb(with local changes)
readium-js@9479ea62e41a3d65860bd0c6e76d924b3ad5e43d
readium-shared-js@8303a0bc1ebc00aceb0c8246a225c436d4e18a3c

[danielweck]

My build version (copy/paste from the Readium cloud reader top-left button dialog):

2.16.0
Wed Jan 21 2015 14:52:40 GMT+0000 (GMT)
readium-js-viewer@b7d480bc10c9faaca2a5dbf3fa7af2a52a9541cb(with local changes)
readium-js@9479ea62e41a3d65860bd0c6e76d924b3ad5e43d
readium-shared-js@8303a0bc1ebc00aceb0c8246a225c436d4e18a3c

[danielweck]

So, to recap (as discussed on the conference call + email exchanges):

Specifically in the case of Chrome (not other WebKit-based web browsers), Readium currently "injects" a CSS 3D nil-transform on video elements, so that they display via a hardware/GPU-accelerated rendering path. Without this "hack", videos that are located in any other CSS column (i.e. reflow page) than the first one, behave badly: inaccessible controls, invisible surface, etc. This is reproducible on both OSX and Windows. However, this seems to depend on codecs, and more importantly: on the machine's GPU/drivers! In fact, we observed that the CSS3D "hack" had the exact opposite effect on someone else's OSX machine (same OS+software version, different hardware). In other words, we solve the problem for some users, but we break video playback for others!

CSS3D "hack":
https://github.com/readium/readium-shared-js/blob/develop/js/views/reflowable_view.js#L231

Note: @rkwright hand-crafted a "reduced" version of the Singapore video EPUB3, but I added a few paragraphs of text before the videos so that they get displayed in the second+ column (i.e. reflow page other than the first one), as the first column usually exhibits the correct behaviour. See Ric's EPUB here:
https://readiumfoundation.box.com/shared/static/jsitratcgydmarb4qoypvrc4k0drh381.epub

Additional observations:

• this EPUB3 works fine in Internet Explorer 11
• in Firefox (Windows) the second column/page blinks and disappears (which a window resize sometimes cures). So it looks like Gecko also freaks out in some cases, because of CSS columns.
• note how Chrome (Windows) displays the poster image using incorrect aspect ratio, regardless of the column/page number.
• with Opera Vivaldi (also WebKit-based) the video controls are totally inaccessible / disabled, not sure why.
• the poster image mismatched aspect ratio is handled differently depending on web browsers (image fit vs. fill etc.).
• the poster image in Chrome (with the CSS3D hack enabled) is briefly displayed, but immediately replaced with the first frame of the video using the poster image's wrong aspect ratio.

[danielweck]

Somewhat related issue (same EPUB3 test is used):
#280

[danielweck]

I posted a Chromium bug report, and I handcrafted a test case (which works fine in IE and Firefox, but fails in Chrome):

https://code.google.com/p/chromium/issues/detail?id=458520

http://www.daisy.org/projects/tobi/VideoCSSColumns/

@rkwright could you please try on your MacBook?

[danielweck]

Update with Chrome 42, and the --enable-region-based-columns experimental flag (or soon to be superseded by --enable-experimental-web-platform-features):

https://code.google.com/p/chromium/issues/detail?id=458520#c9

https://code.google.com/p/chromium/issues/detail?id=421099#c16

(not quite working yet: video rendering improvement, but no visible controls)

[danielweck]

CSS column-fill: auto works with the --enable-experimental-web-platform-features flag in Chrome 42, so this must be used alongside --enable-region-based-columns to achieve the desired result.

[danielweck]

Good news: this is fixed in Chrome 45...well, the video plays and can be interacted with, but it takes some guess work as the video controls are actually not rendered! Still, not as bad a bug as before.
Basically, the Chrome team seem to have decided to activate the aforementioned experimental flags by default in this newer release.

[hihemant84]

Hi danielweck, did you hear back from chrome team about this issue? Have they provided any solution to fix controls hide issue in videos?

[hihemant84]

Hi danielweck, did you hear back from chrome team about this issue? Have they provided any solution to fix controls hide issue in videos?

[danielweck]

@hihemant84 The only recent update is "Renaming Blink>Video to Blink>Media>Video for better characterization" https://bugs.chromium.org/p/chromium/issues/detail?id=458520#c17

[danielweck]

This bug is now fixed in Chrome.
https://bugs.chromium.org/p/chromium/issues/detail?id=458520