Stability checker frequently timing out on Chrome

[gsnedders]

17:17 < jgraham> There's some bug involving chromedriver that we don't understand where sometimes it just refuses to connect

@bobholt is apparently looking into this.

cc/ @jgraham @jugglinmike

[jugglinmike]

@bobholt is looking in to this.

[bobholt]

The relevant PRs to this issue is #5370. 5263 and 5395 are mis-attributed. The Chrome build in 5395 ran all tests to completion in Chrome and produced results. 5263 has a demonstrable error unrelated to timeouts.

The issue that distinguishes this from #5406 is that in this, Chromedriver only tries to start once on each port, but gets a notification of "No output has been received in the last 10m0s, this potentially indicates a stalled build or something wrong with the build itself." at the end.

[bobholt]

Going back through PRs, all PRs since 11:15am EDT on April 7 have completed Chrome tests that match the Firefox tests. I am also no longer able to recreate the timeouts on my fork.

Unless this PR success is due to re-triggering Chrome jobs, I propose that this be closed. This is unsatisfying to me personally, because I was not able to track down a root cause for the timeouts, but as I no longer recreate the behavior, I don't hold out much hope of figuring it out.

[gsnedders]

https://travis-ci.org/w3c/web-platform-tests/jobs/223375148 is a build timing out

[jugglinmike]

@gsnedders That job seems to have completed successfully. Travis reports "Done. Your build exited with 0." and (unlike in recent cases) marks the job as "passed." I checked the other "check stability" job for the same build, and things look to be the same there. Maybe that was the wrong link?

[gsnedders]

@jugglinmike I stupidly mixed things up and hit restart on it for some reason thinking that it got a new job number. :(

[RByers]

Here's a recent example:

PID 6811 | Starting ChromeDriver 2.31.488763 (092de99f48a300323ecf8c2a4e2e7cab51de5ba8) on port 4444
PID 6811 | Only local connections are allowed.
u'log' (u'debug', {'message': 'Hanging up on Selenium session'})
u'runner_teardown' ()
PID 6934 | Starting ChromeDriver 2.31.488763 (092de99f48a300323ecf8c2a4e2e7cab51de5ba8) on port 4444
PID 6934 | Only local connections are allowed.
u'log' (u'debug', {'message': 'Hanging up on Selenium session'})
u'runner_teardown' ()
PID 7058 | Starting ChromeDriver 2.31.488763 (092de99f48a300323ecf8c2a4e2e7cab51de5ba8) on port 4444
PID 7058 | Only local connections are allowed.
u'log' (u'debug', {'message': 'Hanging up on Selenium session'})
u'runner_teardown' ()
...
No output has been received in the last 10m0s, this potentially indicates a stalled build or something wrong with the build itself.

Trying to reproduce.

[RByers]

WebRTC tests seem to reproduce this failure consistently on the bots, though doing a simple wpt run locally seems to work fine for me.

[kereliuk]

@lukebjerring any update on this?

[lukebjerring]

@foolip might have more info on this - think it's something bocoup will tackle

[foolip]

Assigning @mattl. This is not urgent, but something we should try to understand. Matt, can you take a look at this some time over the coming month to see what the cause might be?

[mattl]

Absolutely.

[kereliuk]

@mattl Do you have any update on this issue?

[mattl]

@JKereliuk Afraid not. I wound up getting put onto other things.

[alijuma]

Re-ping about an update since this has priority:roadmap.

[foolip]

@mattl, is the pulls.web-platform-tests.org database possible to extract? I'd like to interrogate it a bit to see how frequently Chrome, Edge and Safari fail. That's to figure out if any of them could be blocking, but it would also tell us if this is still an issue.

[Hexcles]

I think with the various fixes for #9932 , this is no longer an issue.