travis run for chrome unstable (as for now version 54) fails on qx.test.ui.embed Flash:testEvents

[level420]

See https://travis-ci.org/qooxdoo/qooxdoo/jobs/158131683

The log says:

Chrome 54.0.2840 (Linux 0.0.0) qx.test.ui.embed Flash:testEvents FAILED
    expected loaded to have been called at least once but was never called: Called fail().
        qx.core.AssertionError.constructor:43:38
        qx.core.Assert.__fail:71:19
        qx.core.Assert.fail:139:12
        qx.type.BaseString.fail:50:22
        qx.dev.unit.Sinon.failAssertion:4754:20
        anonymous()
        anonymous()
        qx.dev.unit.TestResult.run:237:36
        anonymous()
        anonymous()
        qx.event.dispatch.Direct.prototype.dispatchEvent()
        wrappedFunction()
        qx.event.Manager.prototype.dispatchEvent()
        qx.event.Registration.fireEvent()
        qx.core.Object.prototype.fireEvent()
        anonymous()
        qx.event.Timer.prototype._oninterval()
        anonymous()

[level420]

This may be due to the upcoming changes in flash support for chrome:

https://chrome.googleblog.com/2016/08/flash-and-chrome.html

[level420]

I've run the failing test (and all others) with testrunner on Windows 7 with Chrome 54.0.2840.16 beta-m (distrubuted today via the beta channel) where the tests succeeded.

So it seems to be specific to either the travis environment or the linux version of Chrome.

We should maybe wait for the next beta to see if this is a bug in chrome.

Also note that all other flash related tests work.

[cajus]

Looks like it's no more reproducible locally. I had it yesterday, but now as I want to debug it, the test is green and I can't get it to fail in chrome 45. Doh.

[level420]

@cajus how did you serve? both tests with php server?

[cajus]

yes

[cajus]

So it looks like the tests are enabled, but the PercentLoaded() method returns a TypeError.

[level420]

@cajus is it reproducible again?

[cajus]

only on travis

[cajus]

Ok. It's only on Ubuntu. No idea why I've seen it exactly once on my Fedora installation, though.

On a recent Ubuntu system, it can be fixed by installing the adobe-flashplugin package. As travis' ubuntu is stock old, that will not work and the flashplugin-installer method doesn't seem to install the chromium pepper hooks, so it does not replace the chrome base flash.

Sigh. The travis base system is really frustrating. Looks like I've to install a stock old ubuntu vm to see what flashplugin-installer installs in which location to be able to add the pepper hook.

Edit: it's a bug in the chrome*deb packages

[level420]

@cajus I don't know if it is worth finding out why it doesn't work on travis old ubuntu. Could we maybe detect that we are running on travis and disable this specific test for this environment?

[level420]

It also may be worth considering to use the "TRUSTY BETA" test environment as per documentation https://docs.travis-ci.com/user/ci-environment/#Virtualization-environments if the old ubuntu version is the problem. TRUSTY BETA is v 14 while the standard docker is on v 12.

TRUSTY BETA is slow on startup compared to the docker based environment, but our tests run 8 to 10 minutes anyway, so one minute more does not matter.

[cajus]

@level420 I've described why it fails in the last comment. And I described what could be done to make it work again. Working it around in travis is possible, but it's a waste of time IMHO. Lets just deprecate flash. It's dead. Or wait until google fixes the embedded flash.

[cajus]

@level420 the trusty beta will not help. you can only install whitelisted packages. The ones we need are not whitelisted. So it makes no difference. Or we go back to complete installations instead of docker images - but this is much slower than the current approach. Building a custom docker image is a way to solve it. But I doubt that we'll get it running smoothly in a couple of days. That much too much for one failing test in chrome.

[level420]

OK so I'll create a PR which disables the failing test for chrome on linux.

[level420]

The chrome travis runs succeed again with my PR. See https://travis-ci.org/qooxdoo/qooxdoo/builds/161873683

[level420]

Fixed via #9184