Balancer only works with two hubs

[MissiaL]

MY ENV

5cbfd8521449        aerokube/ggr:latest        "/usr/bin/ggr -lis..."   33 minutes ago      Up 33 minutes                                ggr
e60db9cb01c9        aerokube/selenoid:latest   "/usr/bin/selenoid..."   About an hour ago   Up About an hour    0.0.0.0:4456->4444/tcp   selenoid2
67177dac9cd7        aerokube/selenoid:latest   "/usr/bin/selenoid..."   About an hour ago   Up About an hour    0.0.0.0:4455->4444/tcp   selenoid1
2e104a5cfe62        aerokube/selenoid:latest   "/usr/bin/selenoid..."   About an hour ago   Up About an hour    0.0.0.0:4454->4444/tcp   selenoid

<qa:browsers xmlns:qa="urn:config.gridrouter.qatools.ru">
<browser name="firefox" defaultVersion="52.0">
    <version number="52.0">
        <region name="1">
            <host name="localhost" port="4454" count="1"/>
            <host name="localhost" port="4455" count="1"/>
            <host name="localhost" port="4456" count="1"/>
    </version>
</browser>
</qa:browsers>

When I run the tests, I see that only 2 hubs work from 3. Why does this happen?

[vania-pooh]

@MissiaL this is strange because Ggr randomly distributes requests across hubs. Do you see any SESSION_FAILED messages in ggr logs? Are there any messages related to third host in this log?

[MissiaL]

No, I do not see any error messages. However, I'm sure that 3 hubs are running

[aandryashin]

Ggr is stateless and it uses random numbers to distribute requests, so it is possible to forward request to occupied node.

[vania-pooh]

@MissiaL btw, I see you are using Docker to launch all the stuff. Are ports for all Selenoid containers forwarded correctly?

[vania-pooh]

@MissiaL any updates?

[MissiaL]

Yes, hub ports are configured correctly. There are no errors in the logs. But at the same time there are always only 2 hubs out of three (
I do not understand why only 2 hubs work (

[vania-pooh]

@MissiaL how many parallel Selenium sessions are you requesting? You could try to request 3+ sessions and see that sessions are distributed uniformly. Also couldn't you forget to reload quota information? Just send SIGHUP to Ggr container or process:

# docker kill -s HUP ggr

When reloading quota information Ggr prints XML it actually read to log. Please check that your third host is there and try again.

[MissiaL]

how many parallel Selenium sessions are you requesting?

More than three sessions. I tried to create 20 sessions and they only work with two hubs

When reloading quota information Ggr prints XML it actually read to log. Please check that your third host is there and try again.

I see in log that exists three avialable hub. But only two hubs work

[vania-pooh]

Very interesting. Will investigate.

[MissiaL]

I conducted the experiment. Disconnected two workers' hubs and left only one hub that did not work with ggr. Connected to it directly and run one browser.
I get error:
selenium.common.exceptions.WebDriverException: Message: marionette was not a the name of a known capability or a valid extension capability
I looked at the hub logs

2017/06/13 15:50:25 [NEW_REQUEST]
2017/06/13 15:50:25 [NEW_REQUEST_ACCEPTED]
2017/06/13 15:50:25 [0] [LOCATING_SERVICE] [firefox-]
2017/06/13 15:50:25 Using default version: 53.0
2017/06/13 15:50:25 [0] [USING_DOCKER] [firefox-53.0]
2017/06/13 15:50:25 [0] [CREATING_CONTAINER] [selenoid/firefox:53.0]
2017/06/13 15:50:25 [0] [STARTING_CONTAINER] [selenoid/firefox:53.0] [3ef70bba1241c11cd81d1c0fd5babe129a652cf0399450469be878cc2646330b]
2017/06/13 15:50:25 [0] [CONTAINER_STARTED] [selenoid/firefox:53.0] [3ef70bba1241c11cd81d1c0fd5babe129a652cf0399450469be878cc2646330b] [323.668734ms]
2017/06/13 15:50:25 [0] [SERVICE_STARTED] [selenoid/firefox:53.0] [3ef70bba1241c11cd81d1c0fd5babe129a652cf0399450469be878cc2646330b] [99.375509ms]
2017/06/13 15:50:25 [0] [PROXY_TO] [selenoid/firefox:53.0] [3ef70bba1241c11cd81d1c0fd5babe129a652cf0399450469be878cc2646330b] [http://172.17.0.3:4444/wd/hub]
2017/06/13 15:50:25 [0] [SESSION_ATTEMPTED] [unknown] [http://172.17.0.3:4444/wd/hub] [1]
2017/06/13 15:50:26 [0] [SESSION_FAILED] [unknown] [Bad response from http://172.17.0.3:4444/wd/hub - 400 Bad Request]
2017/06/13 15:50:26 [0] [REMOVE_CONTAINER] [3ef70bba1241c11cd81d1c0fd5babe129a652cf0399450469be878cc2646330b]
2017/06/13 15:50:26 [0] [CONTAINER_REMOVED] [3ef70bba1241c11cd81d1c0fd5babe129a652cf0399450469be878cc2646330b]

Maybe it to help you

Afetr them i set version to 52.0 and restart test. The test successfully worked

[MissiaL]

If I configure all three hubs to connect to only one version of Firefox, then there are still only two hubs working (

[vania-pooh]

@MissiaL I reproduced the issue with test. Will be fixed soon.

[vania-pooh]

Fixed in #80. Please try to run your tests against: aerokube/ggr:latest (will be released soon).