Stabilize blocklayout.feature #26148
Conversation
| # Do our async wait on the JavaScript side instead of polling over the wire. | ||
| # See https://seleniumhq.github.io/selenium/docs/api/java/org/openqa/selenium/JavascriptExecutor.html | ||
| # and https://www.rubydoc.info/gems/selenium-webdriver/3.8.0/Selenium/WebDriver/Driver#execute_async_script-instance_method | ||
| result = @browser.execute_async_script <<-JS |
There was a problem hiding this comment.
We haven't used execute_async_script before. It passes a callback as the last argument to your JavaScript, and blocks until the callback is called (or the selenium script timeout is reached).
The idea here is that we don't need our test runner to poll SauceLabs waiting for something to be available - we can do that in injected JavaScript on the client and call back when ready. More generally, I wish we used fewer over-the-wire commands per Cucumber step, and I think this is a step in the right direction.
| } | ||
| }()) | ||
| JS | ||
| expect(result).to be_nil |
There was a problem hiding this comment.
Note, although there's a Selenium script timeout, it gives a spew of confusing output when you hit it. I'm implementing my own client-side timeout here (5 seconds seemed pretty good in this case where we weren't waiting at all before) that will result in a nice rspec assertion failures if it's reached.
| @@ -58,6 +58,9 @@ def saucelabs_browser(test_run_name) | |||
| http_client: http_client | |||
| ) | |||
|
|
|||
| # Maximum time a single execute_script or execute_async_script command may take | |||
| browser.manage.timeouts.script_timeout = 30.seconds | |||
There was a problem hiding this comment.
I'm not sure where the default was coming from, but without this line I was getting
Timeout expired waiting for async script
Command duration or timeout: 98 milliseconds
Elijah noticed significant flakiness in this test today. After a little investigation I decided the safest thing is to wait for
Blockly.mainBlockSpaceto be available on the client before we try to clear it.Ran against IE11 three times in a row without any scenarios failing, and then ran against our whole set of browsers and everything passed on the first try.
I'll make some notes inline about my approach here (it's somewhat new-to-us) and I'd like to follow this with some more cleanup to our steps, but figured getting the initial stability fix in was important.