Based on an informal audit of Travis builds (i.e. me pointing and clicking to restart tests for the past few days), our most faily tests seem to be iPhone and IE. If we can find ways to make these tests less likely to fail I would be super happy.
Some stuff I'm trying out:
I would really love if our test suite were reliably green/red rather than being nearly guaranteed to have a false negative on any given run.
Occasionally I also see No output has been received in the last 10m0s for the Firefox builds. I believe this is due to not being able to find the installed Firefox 48.0.
No output has been received in the last 10m0s
(#5930) - use yarn
(#5930) - use default Firefox 37 in Travis
(#5930) - WIP - test firefox
(#5930) - test Firefox 50 in Travis
Might be related to #5898, but IE10+Win8.1 tests are super flaky these days. I might have to deep-dive to see what's up.
Wow, our build is really trolling us right now. On a typical run I have to restart like 6 jobs.
OK, I'm fairly certain 4eb786b regressed us on IE10. Tests rarely seem to want to pass for that browser now.
(#5930) - simplify setTimeout/nextTick/immediate usage
(#5930) - use Firefox 41.0.1 in Travis
(#5930) - use Firefox 49.0.2 in Travis
(#5930) - use Firefox 46.0.1 in Travis, add FIREFOX_BIN
(#5930) - run Firefox 50 in Travis
(#5930) - use Firefox 47.0.2 for all Firefox tests
(#5930) - cache selenium and phantomjs in Travis
(#5908 #5930) - cache CouchDB 1.6.1 Docker image on Travis
Looking into this now, firefox seems far more faily than others, however I am seeing a fair amount of fails across the board as well so doing a large travis run @ #5967 and once we know / can get node stable then ill look at firefox
I think it's reasonable to close this now. Done a lot of work here recently.
Revert "(#5930) - cache selenium and phantomjs in Travis"
This reverts commit 43ca134.
(#5930) - use travis_retry