Grid's Hub reports to have capacity when node is marked down #6494
Description
Meta
Selenium Version:
The one available in selenium/hub:3.14.0-gallium
Expected Behavior
When all the nodes in the Hub have been marked down, the Hub should report it does not have capacity.
Actual Behavior -
The Hub marks the node as down immediately after closing the node, however it keeps reporting to have capacity for 50+ seconds.
Steps to reproduce -
Have a Hub running with no nodes. Check how it reports that it does not have capacity.
Now add a browser, for example a Chrome. The Hub now reports to have capacity.
Kill the browser. The Hub's log has an entry like Marking the node http://<IP>:5555 as down: cannot reach the node for 2 tries, however, it keeps reporting to have capacity.
A while later, the Hub's log will have Cleaning up stale test sessions on the unregistered node http://<IP>:5555, and will correctly report that it has no capacity.
Actual commands to reproduce it in a single computer (can remove remote_host and share a docker network or whatever). Assume my computer IP is 192.168.1.2.
Open 3 tabs:
Tab 1: docker run -it -p 4444:4444 selenium/hub:3.14
Tab 2: watch -n 1 curl http://localhost:4444/wd/hub/status
No capacity.
Tab 3: docker run -it -p 5555:5555 -e "HUB_HOST=192.168.1.2" -e "HUB_PORT=4444" -e REMOTE_HOST="http://192.168.1.2:5555" --shm-size=2g selenium/node-chrome:3.14
Tab 2: reports capacity
Tab 3: CTRL + C
Tab 1: Marking the node http://192.168.1.2:5555 as down: cannot reach the node for 2 tries
Tab 3: reports capacity -----> WRONG! Here the Hub should report it does not have capacity
Wait around 50 seconds...
Tab 1: Cleaning up stale test sessions on the unregistered node http://192.168.1.2:5555
Tab 3: reports it does not have capacity