Grid Stuck When > ~500 Tests Are Executed In Parallel With ChromeDriver

[ryanlevell]

This became an issue sometime after version 2.47.1.
I tested two versions, 2.48.2 and 2.52.0 and they both get stuck.

I duplicated this issue locally using a single computer with a grid running 5 instances. My real grid environment has 100 browser instances available, hence why I sometimes have 500 tests queued.

I am using ChromeDriver 2.21.

Expected Behavior

Each test iteration opens a new Chrome browser and runs the test normally.

Actual Behavior

The first set of browsers get stuck on the initial data:, screen in Chrome.

Steps to reproduce

1. Start the hub
   • No Issue: java -jar selenium-server-standalone-2.47.1.jar -role hub
   • Issue: java -jar selenium-server-standalone-2.52.0.jar -role hub
2. Start a node
   • No Issue: java -jar selenium-server-standalone-2.47.1.jar -role node -hub http://localhost:4444/grid/register -Dwebdriver.chrome.driver=c:\selenium_drivers\chromedriver.exe
   • Issue: java -jar selenium-server-standalone-2.52.0.jar -role node -hub http://localhost:4444/grid/register -Dwebdriver.chrome.driver=c:\selenium_drivers\chromedriver.exe
3. Run the test
   • mvn clean test
   • Three files below

testng.xml

<suite name="Suite" data-provider-thread-count="100" parallel="methods" thread-count="10">
    <test name="Test">
        <classes>
            <class name="com.mytest.AppTest"/>
        </classes>
    </test>
</suite>

src/test/java/com/mytest/AppTest.java

package com.mytest;

import java.net.MalformedURLException;
import java.net.URL;

import org.openqa.selenium.WebDriver;
import org.openqa.selenium.remote.DesiredCapabilities;
import org.openqa.selenium.remote.RemoteWebDriver;
import org.testng.annotations.DataProvider;
import org.testng.annotations.Test;

public class AppTest {

    @Test(dataProvider = "dp")
    public void test1() throws MalformedURLException {
        DesiredCapabilities caps = DesiredCapabilities.chrome();
        WebDriver driver = new RemoteWebDriver(new URL("http://localhost:4444/wd/hub"), caps);
        driver.get("http://google.com");
        driver.quit();
    }

    @Test(dataProvider = "dp")
    public void test2() throws MalformedURLException {
        DesiredCapabilities caps = DesiredCapabilities.chrome();
        WebDriver driver = new RemoteWebDriver(new URL("http://localhost:4444/wd/hub"), caps);
        driver.get("http://google.com");
        driver.quit();
    }

    @Test(dataProvider = "dp")
    public void test3() throws MalformedURLException {
        DesiredCapabilities caps = DesiredCapabilities.chrome();
        WebDriver driver = new RemoteWebDriver(new URL("http://localhost:4444/wd/hub"), caps);
        driver.get("http://google.com");
        driver.quit();
    }

    @DataProvider(name = "dp", parallel = true)
    public static Object[][] dp() {
        return new Object[500][0];
    }
}

pom.xml

<project>
    <modelVersion>4.0.0</modelVersion>
    <groupId>com.mytest</groupId>
    <artifactId>ManyTests</artifactId>
    <version>0.0.1-SNAPSHOT</version>

    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-surefire-plugin</artifactId>
                <version>2.19.1</version>
                <configuration>
                    <suiteXmlFiles>
                        <suiteXmlFile>testng.xml</suiteXmlFile>
                    </suiteXmlFiles>
                </configuration>
            </plugin>
        </plugins>
    </build>

    <dependencies>
        <dependency>
            <groupId>org.testng</groupId>
            <artifactId>testng</artifactId>
            <version>6.9.10</version>
        </dependency>
        <dependency>
            <groupId>org.seleniumhq.selenium</groupId>
            <artifactId>selenium-server</artifactId>
            <version>2.52.0</version>
        </dependency>
        <dependency>
            <groupId>org.seleniumhq.selenium</groupId>
            <artifactId>selenium-java</artifactId>
            <version>2.52.0</version>
        </dependency>
    </dependencies>
</project>

[jimpatel90]

is there any updates on this? we are facing similar issue.

[bdabburi3091]

is there any updates on this? we are facing similar issue on 2.53.0 version also

[leogcba]

I'm having the similar issue using grid 3.14

These are my settings for the hub:

{
  "port": 4444,
  "newSessionWaitTimeout": 60000,
  "servlets" : [],
  "withoutServlets": [],
  "custom": {},
  "capabilityMatcher": "org.openqa.grid.internal.utils.DefaultCapabilityMatcher",
  "registry": "org.openqa.grid.internal.DefaultGridRegistry",
  "throwOnCapabilityNotPresent": true,
  "cleanUpCycle": 5000,
  "role": "hub",
  "debug": false,
  "browserTimeout": 60,
  "timeout": 600
}

These are my settings for a node (This setup is just to perform a load test against the grid):

{
  "capabilities":
  [
    {
      "browserName": "chrome",
      "maxInstances": 1,
      "seleniumProtocol": "WebDriver"
    }
  ],
  "maxSession": 1,
  "port": 4445,
  "register": true,
  "registerCycle": 5000,
  "hubHost": "192.168.1.123",
  "hubPort": 4444,
  "nodeStatusCheckTimeout": 5000,
  "nodePolling": 5000,
  "role": "node",
  "unregisterIfStillDownAfter": 60000,
  "downPollingLimit": 3
}

I run 6 parallel tests and the first 2 or 3 tests from the queue execute fine. But when the queue starts to grow, all the remaining tests are added to the queue and start to fail.

Watching at the browser in the node, the loaded URL just says data; and no more tests execute. The queue gets bigger and bigger.

Finally, the queue starts to drop the pending sessions until it gets empty. At this moment, the browser closes and is ready to work with new tests.

If I start to run 6 suites of tests in parallel, the behavior is the same. The firsts tests in the queue run ok, then the node gets locked.

I think this might be related with #3022

Maybe there's something wrong with the values for the timeouts?

Thoughts?

[cgoldberg]

you have maxSession and maxInstances set to 1.

[diemol]

This issue got stale and no more information was provided.
Since Grid 2.x are 3.x are not developed anymore, if someone runs into a similar issue with Grid 4.x (in alphas currently), please create a new issue with the complete template filled out.

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.