Unexpected hanging of the Loader after actions

[SkorikSergey]

Description

We have unexpected fail of FileStructureNodesTest selenium test. The test sometimes fails while is waiting for the File Structure form.

Reproduction Steps:

• Open a java class in the Editor.
• Press Ctrl+F12 or menu Assistant -> Navigate File Structure.

Expected behavior:
The File Structure form is opened.

Observed behavior:
The File Structure form is not appeared and the test fails because the Loader is still visible(after waiting for 180 second that it is closed).

Failed selenium test: FileStructureNodesTest, PreviewUrlIntoCommandsEditorTest

Log:

org.openqa.selenium.TimeoutException: 
Expected condition failed: waiting for element to no longer be visible: By.id: gwt-debug-loader-panel (tried for 180 second(s) with 500 MILLISECONDS interval)
Build info: version: '3.0.1', revision: '1969d75', time: '2016-10-18 09:49:13 -0700'
System info: host: 'slave4.codenvycorp.com', ip: '127.0.0.1', os.name: 'Linux', os.arch: 'amd64', os.version: '3.10.0-514.16.1.el7.x86_64', java.version: '1.8.0_102'
Driver info: org.eclipse.che.selenium.core.SeleniumWebDriver
	at org.eclipse.che.selenium.miscellaneous.FileStructureNodesTest.checkFileStructureNodes(FileStructureNodesTest.java:127)

Screenshot:

Report with reproduced problem:
https://ci.codenvycorp.com/view/qa/job/che-integration-tests-master-docker/148/Selenium_tests_report/

[SkorikSergey]

Similar behavior with RenameParametersTest, TestAnnotationsTest:

Log:

org.openqa.selenium.TimeoutException: 
Expected condition failed: waiting for element to no longer be visible: By.id: gwt-debug-loader-panel (tried for 180 second(s) with 500 MILLISECONDS interval)
Build info: version: '3.0.1', revision: '1969d75', time: '2016-10-18 09:49:13 -0700'
System info: host: 'slave6.codenvycorp.com', ip: '127.0.0.1', os.name: 'Linux', os.arch: 'amd64', os.version: '3.10.0-514.16.1.el7.x86_64', java.version: '1.8.0_102'
Driver info: org.eclipse.che.selenium.core.SeleniumWebDriver
	at org.eclipse.che.selenium.refactor.parameters.RenameParametersTest.checkRenameParameters0(RenameParametersTest.java:75)

https://ci.codenvycorp.com/view/qa/job/che-integration-tests-master-docker/157/Selenium_tests_report/

Screenshot:

[vparfonov]

I've add some messages to the loader it will help identify problems more accurate. Not close issue yet will wait on detail information

[SkorikSergey]

Screenshot after adding messages to the Loader:

[Ohrimenko1988]

Attachment for PreviewUrlIntoCommandsEditorTest

Log:

org.openqa.selenium.WebDriverException: 
unknown error: Element <button type="button" class="gwt-Button GEU2T3BBGJC" id="gwt-debug-command-editor-button-run">...</button> is not clickable at point (1869, 103). Other element would receive the click: <div id="gwt-debug-glass-panel" class="GEU2T3BBCCB">...</div>
  (Session info: chrome=56.0.2924.87)
  (Driver info: chromedriver=2.27.440175 (9bc1d90b8bfa4dd181fbbf769a5eb5e575574320),platform=Linux 3.10.0-514.16.1.el7.x86_64 x86_64) (WARNING: The server did not provide any stacktrace information)
Command duration or timeout: 165 milliseconds
Build info: version: '3.1.0', revision: '86a5d70', time: '2017-02-16 07:57:44 -0800'
System info: host: '251773b88214', ip: '172.22.0.5', os.name: 'Linux', os.arch: 'amd64', os.version: '3.10.0-514.16.1.el7.x86_64', java.version: '1.8.0_121'
Driver info: org.openqa.selenium.chrome.ChromeDriver
Capabilities [{applicationCacheEnabled=false, rotatable=false, mobileEmulationEnabled=false, networkConnectionEnabled=false, chrome={chromedriverVersion=2.27.440175 (9bc1d90b8bfa4dd181fbbf769a5eb5e575574320), userDataDir=/tmp/.org.chromium.Chromium.tJaSOB}, takesHeapSnapshot=true, pageLoadStrategy=normal, databaseEnabled=false, handlesAlerts=true, hasTouchScreen=false, version=56.0.2924.87, platform=LINUX, browserConnectionEnabled=false, nativeEvents=true, acceptSslCerts=true, locationContextEnabled=true, webStorageEnabled=true, browserName=chrome, takesScreenshot=true, javascriptEnabled=true, cssSelectorsEnabled=true, unexpectedAlertBehaviour=}]
Session ID: ee26f4cfc9871c324bc78137d1e5bc93
Command duration or timeout: 830 milliseconds
Build info: version: '3.0.1', revision: '1969d75', time: '2016-10-18 09:49:13 -0700'
System info: host: 'slave2.codenvycorp.com', ip: '127.0.0.1', os.name: 'Linux', os.arch: 'amd64', os.version: '3.10.0-514.16.1.el7.x86_64', java.version: '1.8.0_102'
Driver info: org.openqa.selenium.remote.RemoteWebDriver
Capabilities [{applicationCacheEnabled=false, rotatable=false, mobileEmulationEnabled=false, networkConnectionEnabled=true, chrome={chromedriverVersion=2.27.440175 (9bc1d90b8bfa4dd181fbbf769a5eb5e575574320), userDataDir=/tmp/.org.chromium.Chromium.tJaSOB}, takesHeapSnapshot=true, pageLoadStrategy=normal, databaseEnabled=false, handlesAlerts=true, hasTouchScreen=true, version=56.0.2924.87, platform=LINUX, browserConnectionEnabled=false, nativeEvents=true, acceptSslCerts=true, webdriver.remote.sessionid=9ca65426-648a-4895-9fa1-f2a2157b2463, locationContextEnabled=true, webStorageEnabled=true, browserName=chrome, takesScreenshot=true, javascriptEnabled=true, cssSelectorsEnabled=true, unexpectedAlertBehaviour=}]
Session ID: 9ca65426-648a-4895-9fa1-f2a2157b2463
	at org.eclipse.che.selenium.intelligencecommand.PreviewUrlIntoCommandsEditorTest.checkSavePreviewUrlIntoCommand(PreviewUrlIntoCommandsEditorTest.java:89)

Screenshot:

[dmytro-ndp]

It is reliably reproduced when CPU is overloaded, for example, when the number of test threads >= number of CPU cores.
For the qa-slaves it is --threads=4 and more: https://ci.codenvycorp.com/view/qa-experimental/job/experimental-che-integration-tests-master-docker/88/Selenium_tests_report/

Report from here:

1. Webdriver log: org.eclipse.che.selenium.refactor.types.TestAnnotationsTest.testAnnotation1_kuf2.log.
2. Ws agent logs:

• catalina.log
• localhost-access.log

According to the logs there was a problem with processing request:

POST http://172.30.3.252:43078/api/java/refactoring/rename/create

[tsmaeder]

The try/catch blocks for FileStructureNodesTest, at least are too broad:

   try {
      fileStructure.waitExpectedTextInFileStructure(ITEMS_CLASS);
    } catch (WebDriverException ex) {
      // remove try-catch block after issue has been resolved
      fail("Known issue https://github.com/eclipse/che/issues/8300");
    }

I have at least one case where the problem is not a hung loader, but the expected text is wrong (looking at the screenshot). If we need to preserve context, we should pass it into the FileStructure.wait...() methods.

[vzhukovs]

Investigation has shown that the issue can't be reproduced neither on jenkins nor local machine.