Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

JDKHttpClient: Handle RQST with no Body [POST/PUT] #11445

Merged
merged 1 commit into from
Dec 20, 2022

Conversation

krmahadevan
Copy link
Contributor

Fixes: #11342

Thanks for contributing to Selenium!
A PR well described will help maintainers to quickly review and merge it

Before submitting your PR, please check our contributing guidelines.
Avoid large PRs, help reviewers by making them as simple and short as possible.

Description

Here's what I have as analysis:

  1. We got rid of all places wherein we are setting the content length explicitly to zero.
  2. When starting a container, we use the Docker container API with a POST message, but it has no body and so its content length now gets evaluated to zero.
  3. We are using the JDK client by default for all of our http client needs.
  4. We create a HttpRequest via org.openqa.selenium.remote.http.jdk.JdkHttpMessages#createRequest which internally calls org.openqa.selenium.remote.http.jdk.JdkHttpMessages#notChunkingBodyPublisher for a POST message.
  5. Since as per the docker api spec, there's no payload for this request, we end up getting evaluated to a content length of zero.
  6. This in turn triggers the validation check at the JDK HttpClient level saying
grid-node-docker-1    | java.lang.IllegalArgumentException: non-positive contentLength: 0
grid-node-docker-1    |         at java.net.http/java.net.http.HttpRequest$BodyPublishers.fromPublisher(HttpRequest.java:539)
grid-node-docker-1    |         at org.openqa.selenium.remote.http.jdk.JdkHttpMessages.notChunkingBodyPublisher(JdkHttpMessages.java:124)
grid-node-docker-1    |         at org.openqa.selenium.remote.http.jdk.JdkHttpMessages.createRequest(JdkHttpMessages.java:76)
grid-node-docker-1    |         at org.openqa.selenium.remote.http.jdk.JdkHttpClient.execute(JdkHttpClient.java:292)

Note:
We cannot set a payload also for this request, since when we do that, I hit an error as below

{
    "message": "starting container with non-empty request body was deprecated since API v1.22 and removed in v1.24"
}

with a 400 BAD REQUEST

This PR fixes this discrepancy by ensuring that when we are confronted with empty payloads for POST|PUT messages and when we are using the JDK provided HTTP Client, we resort to using an empty body

Motivation and Context

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)

Checklist

  • I have read the contributing document.
  • My change requires a change to the documentation.
  • I have updated the documentation accordingly.
  • I have added tests to cover my changes.
  • All new and existing tests passed.

@krmahadevan
Copy link
Contributor Author

@diemol @pujagani

Here's how I have tested this

  1. Build the JDK http client by running bazel build //java/src/org/openqa/selenium/remote/http/jdk:jdk-lib
  2. cloned the docker selenium project
  3. Copied /bazel-bin/java/src/org/openqa/selenium/remote/http/jdk/libjdk-lib.jar into docker-selenium/Base/ as selenium-http-jdk-client.jar
  4. Altered the docker-selenium/Base/Dockerfile with below changes
    • commented out wget --no-verbose https://repo1.maven.org/maven2/org/seleniumhq/selenium/selenium-http-jdk-client/4.7.2/selenium-http-jdk-client-4.7.2.jar -O /opt/selenium/selenium-http-jdk-client.jar
    • Added COPY ./selenium-http-jdk-client.jar /opt/selenium/selenium-http-jdk-client.jar
  5. Built all the images by running VERSION=local make build
  6. Altered the docker compose file and the config.toml (references of them are available in the defect ) to refer to the local images.
  7. Started the grid via docker compose
version: "3"
services:
  node-docker:
    image: selenium/node-docker:local-20221213
    volumes:
      - ./config.toml:/opt/bin/config.toml
      - ./assets:/opt/selenium/assets

    depends_on:
      - selenium-hub-dynamic
    environment:
      - SE_EVENT_BUS_HOST=selenium-hub-dynamic
      - SE_EVENT_BUS_PUBLISH_PORT=4442
      - SE_EVENT_BUS_SUBSCRIBE_PORT=4443
    ports:
      - "5005:5005"
     
  selenium-hub-dynamic:
    image: selenium/hub:local-20221213
    container_name: selenium-hub-dynamic
    ports:
      - "4442:4442"
      - "4443:4443"
      - "4444:4444"

config.toml contents

[docker]
configs = [
    "selenium/standalone-firefox:local-20221213","{\"browserName\": \"firefox\"}"
    ]
# Windows: make sure Docker Desktop exposes the daemon via tcp, and use http://host.docker.internal:2375.
# macOS: install socat and run the following command, socat -4 TCP-LISTEN:2375,fork UNIX-CONNECT:/var/run/docker.sock,
# then use http://host.docker.internal:2375.
# Linux: varies from machine to machine, please mount /var/run/docker.sock. If this does not work, please create an issue.
# I am on mac
url =  "http://host.docker.internal:2375"
  1. Ran a sample test that looks like below
import java.net.MalformedURLException;
import org.openqa.selenium.firefox.FirefoxOptions;
import org.openqa.selenium.remote.RemoteWebDriver;
import java.net.URL;

public class Sample {

  public static void main(String[] args) throws MalformedURLException {
    RemoteWebDriver driver = null;
    try {
      driver = new RemoteWebDriver(new URL("http://localhost:4444"), new FirefoxOptions());
      driver.get("http://www.google.com");
      System.err.println("Title " + driver.getTitle());
    } finally {
      if (driver != null) {
        driver.quit();
      }
    }
  }
}

@pujagani
Copy link
Contributor

Logic-wise LGTM.

@diemol diemol force-pushed the handle_http_posts_with_no_body branch from 3707a56 to ccf6278 Compare December 20, 2022 09:54
@diemol diemol force-pushed the handle_http_posts_with_no_body branch from ccf6278 to 6fd1537 Compare December 20, 2022 10:56
Copy link
Member

@diemol diemol left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you, @krmahadevan!

@diemol diemol merged commit 83d3d46 into SeleniumHQ:trunk Dec 20, 2022
@krmahadevan krmahadevan deleted the handle_http_posts_with_no_body branch December 21, 2022 05:20
@jsa34
Copy link

jsa34 commented Dec 21, 2022

@diemol I'm assuming this is still not in the latest docker images (4.7.2-20221219) as I am still seeing on dynamic grid:

Hub:

09:09:30.479 WARN [SeleniumSpanExporter$1.lambda$export$1] - Unable to create session: Could not start a new session. Could not start a new session. non-positive contentLength: 0
Host info: host: '10949db76ec3', ip: '0.0.0.0'
Build info: version: '4.7.2', revision: '4d4020c3b7'
System info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.14.0-162.6.1.el9_1.x86_64', java.version: '11.0.17'
Driver info: driver.version: unknown
Build info: version: '4.7.2', revision: '4d4020c3b7'
System info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.14.0-162.6.1.el9_1.x86_64', java.version: '11.0.17'
Driver info: driver.version: unknown
09:09:30.479 WARN [SeleniumSpanExporter$1.lambda$export$1] - org.openqa.selenium.SessionNotCreatedException: Could not start a new session. Could not start a new session. non-positive contentLength: 0
Host info: host: '10949db76ec3', ip: '0.0.0.0'
Build info: version: '4.7.2', revision: '4d4020c3b7'
System info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.14.0-162.6.1.el9_1.x86_64', java.version: '11.0.17'
Driver info: driver.version: unknown
Build info: version: '4.7.2', revision: '4d4020c3b7'
System info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.14.0-162.6.1.el9_1.x86_64', java.version: '11.0.17'
Driver info: driver.version: unknown
        at org.openqa.selenium.grid.node.remote.RemoteNode.newSession(RemoteNode.java:150)
        at org.openqa.selenium.grid.distributor.local.LocalDistributor.startSession(LocalDistributor.java:645)
        at org.openqa.selenium.grid.distributor.local.LocalDistributor.newSession(LocalDistributor.java:564)
        at org.openqa.selenium.grid.distributor.local.LocalDistributor$NewSessionRunnable.handleNewSessionRequest(LocalDistributor.java:818)
        at org.openqa.selenium.grid.distributor.local.LocalDistributor$NewSessionRunnable.lambda$run$1(LocalDistributor.java:779)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:829)

Node:

09:09:30.468 WARN [SessionSlot.apply] - Unable to create session
java.lang.IllegalArgumentException: non-positive contentLength: 0
        at java.net.http/java.net.http.HttpRequest$BodyPublishers.fromPublisher(HttpRequest.java:539)
        at org.openqa.selenium.remote.http.jdk.JdkHttpMessages.notChunkingBodyPublisher(JdkHttpMessages.java:124)
        at org.openqa.selenium.remote.http.jdk.JdkHttpMessages.createRequest(JdkHttpMessages.java:76)
        at org.openqa.selenium.remote.http.jdk.JdkHttpClient.execute(JdkHttpClient.java:292)
        at org.openqa.selenium.remote.tracing.TracedHttpClient.execute(TracedHttpClient.java:55)
        at org.openqa.selenium.docker.v1_41.StartContainer.apply(StartContainer.java:40)
        at org.openqa.selenium.docker.v1_41.V141Docker.startContainer(V141Docker.java:115)
        at org.openqa.selenium.docker.Container.start(Container.java:47)

Any timescales for release?

@580
Copy link

580 commented Jan 3, 2023

After upgraded to 4.7.2-20221219, I came across to this issue.

@skylarmb
Copy link

skylarmb commented Jan 4, 2023

+1, either this did not get released in 4.7.2-20221219 or this is still broken

@krmahadevan
Copy link
Contributor Author

Folks, just to clarify.

This PR was merged on Dec 20, 2022 and 4.7.2-20221219 was created before that. So I think we would need to wait for a new image to be created for this fix to be available. First Selenium 4.7.3 (or) a higher version would need to be released and then we would need the new image to be created and released so that this fix can be consumed.

@pujagani
Copy link
Contributor

pujagani commented Jan 4, 2023

We have a 4.8 released plan soon (in about a week or so).

@itkhanz
Copy link

itkhanz commented Jan 8, 2023

Hi,
I am getting following error with version 4.7.2-20221219. Is this error because of this issue? or is there someting wrong with my configurations? Could you please have a look at the error logs and suggest fix. Thank you.

Below are the error logs and my dynamic-grid yml file as well as config.toml

selenium-hub

2023-01-09 00:34:27 2023-01-08 23:34:27,677 INFO Included extra file "/etc/supervisor/conf.d/selenium-grid-hub.conf" during parsing
2023-01-09 00:34:27 2023-01-08 23:34:27,689 INFO RPC interface 'supervisor' initialized
2023-01-09 00:34:27 2023-01-08 23:34:27,689 CRIT Server 'unix_http_server' running without any HTTP authentication checking
2023-01-09 00:34:27 2023-01-08 23:34:27,690 INFO supervisord started with pid 9
2023-01-09 00:34:28 2023-01-08 23:34:28,694 INFO spawned: 'selenium-grid-hub' with pid 11
2023-01-09 00:34:28 Tracing is disabled
2023-01-09 00:34:28 2023-01-08 23:34:28,704 INFO success: selenium-grid-hub entered RUNNING state, process has stayed up for > than 0 seconds (startsecs)
2023-01-09 00:34:30 23:34:30.175 INFO [LoggingOptions.configureLogEncoding] - Using the system default encoding
2023-01-09 00:34:30 23:34:30.189 INFO [OpenTelemetryTracer.createTracer] - Using OpenTelemetry for tracing
2023-01-09 00:34:30 23:34:30.459 INFO [BoundZmqEventBus.<init>] - XPUB binding to [binding to tcp://*:4442, advertising as tcp://172.24.0.2:4442], XSUB binding to [binding to tcp://*:4443, advertising as tcp://172.24.0.2:4443]
2023-01-09 00:34:30 23:34:30.777 INFO [UnboundZmqEventBus.<init>] - Connecting to tcp://172.24.0.2:4442 and tcp://172.24.0.2:4443
2023-01-09 00:34:30 23:34:30.896 INFO [UnboundZmqEventBus.<init>] - Sockets created
2023-01-09 00:34:31 23:34:31.901 INFO [UnboundZmqEventBus.<init>] - Event bus ready
2023-01-09 00:34:34 23:34:34.400 INFO [Hub.execute] - Started Selenium Hub 4.7.2 (revision 4d4020c3b7): http://172.24.0.2:4444
2023-01-09 00:34:37 23:34:37.214 INFO [Node.<init>] - Binding additional locator mechanisms: relative, id, name
2023-01-09 00:34:38 23:34:38.236 INFO [GridModel.setAvailability] - Switching Node f34c8932-826b-4e0f-9671-dd355d2e0549 (uri: http://172.24.0.3:5555) from DOWN to UP
2023-01-09 00:34:38 23:34:38.239 INFO [LocalDistributor.add] - Added node f34c8932-826b-4e0f-9671-dd355d2e0549 at http://172.24.0.3:5555. Health check every 120s
2023-01-09 00:40:22 23:40:22.245 WARN [SeleniumSpanExporter$1.lambda$export$3] - {"traceId": "828df50fadd125fd657b29ce777edf82","eventTime": 1673221222249126100,"eventName": "HTTP request execution complete","attributes": {"http.flavor": 1,"http.handler_class": "org.openqa.selenium.grid.sessionqueue.local.LocalNewSessionQueue","http.host": "localhost:4444","http.method": "POST","http.request_content_length": "503","http.scheme": "HTTP","http.status_code": 500,"http.target": "\u002fsession","http.user_agent": "selenium\u002f4.6.0 (java windows)"}}
2023-01-09 00:40:22 
2023-01-09 00:40:22 23:40:22.253 WARN [SeleniumSpanExporter$1.lambda$export$3] - {"traceId": "9bacf15e28cbe4113d08c2a205f8b9e4","eventTime": 1673221222257051100,"eventName": "HTTP request execution complete","attributes": {"http.flavor": 1,"http.handler_class": "org.openqa.selenium.grid.sessionqueue.local.LocalNewSessionQueue","http.host": "localhost:4444","http.method": "POST","http.request_content_length": "487","http.scheme": "HTTP","http.status_code": 500,"http.target": "\u002fsession","http.user_agent": "selenium\u002f4.6.0 (java windows)"}}
2023-01-09 00:40:22 
2023-01-09 00:40:22 23:40:22.279 WARN [SeleniumSpanExporter$1.lambda$export$3] - {"traceId": "d72f78252b083c8a3fc5364b8fdc862d","eventTime": 1673221222282610000,"eventName": "HTTP request execution complete","attributes": {"http.flavor": 1,"http.handler_class": "org.openqa.selenium.grid.sessionqueue.local.LocalNewSessionQueue","http.host": "localhost:4444","http.method": "POST","http.request_content_length": "503","http.scheme": "HTTP","http.status_code": 500,"http.target": "\u002fsession","http.user_agent": "selenium\u002f4.6.0 (java windows)"}}
2023-01-09 00:40:22 
2023-01-09 00:40:22 23:40:22.248 WARN [SeleniumSpanExporter$1.lambda$export$3] - {"traceId": "7f78cd1adba8835e69b0a759deb4000f","eventTime": 1673221222251986800,"eventName": "HTTP request execution complete","attributes": {"http.flavor": 1,"http.handler_class": "org.openqa.selenium.grid.sessionqueue.local.LocalNewSessionQueue","http.host": "localhost:4444","http.method": "POST","http.request_content_length": "493","http.scheme": "HTTP","http.status_code": 500,"http.target": "\u002fsession","http.user_agent": "selenium\u002f4.6.0 (java windows)"}}
2023-01-09 00:40:22 
2023-01-09 00:40:22 23:40:22.241 WARN [SeleniumSpanExporter$1.lambda$export$3] - {"traceId": "49b1423720f3f55107fa1b6701f02fb4","eventTime": 1673221222245418500,"eventName": "HTTP request execution complete","attributes": {"http.flavor": 1,"http.handler_class": "org.openqa.selenium.grid.sessionqueue.local.LocalNewSessionQueue","http.host": "localhost:4444","http.method": "POST","http.request_content_length": "493","http.scheme": "HTTP","http.status_code": 500,"http.target": "\u002fsession","http.user_agent": "selenium\u002f4.6.0 (java windows)"}}
2023-01-09 00:40:22 
2023-01-09 00:40:22 23:40:22.302 WARN [SeleniumSpanExporter$1.lambda$export$3] - {"traceId": "bcfabe2ab36dc126db910ba08e55d795","eventTime": 1673221222306281500,"eventName": "HTTP request execution complete","attributes": {"http.flavor": 1,"http.handler_class": "org.openqa.selenium.grid.sessionqueue.local.LocalNewSessionQueue","http.host": "localhost:4444","http.method": "POST","http.request_content_length": "493","http.scheme": "HTTP","http.status_code": 500,"http.target": "\u002fsession","http.user_agent": "selenium\u002f4.6.0 (java windows)"}}
2023-01-09 00:40:22 
2023-01-09 00:40:22 23:40:22.292 WARN [SeleniumSpanExporter$1.lambda$export$3] - {"traceId": "25a879dea81afd53fa9610f1fd62ad91","eventTime": 1673221222295369800,"eventName": "HTTP request execution complete","attributes": {"http.flavor": 1,"http.handler_class": "org.openqa.selenium.grid.sessionqueue.local.LocalNewSessionQueue","http.host": "localhost:4444","http.method": "POST","http.request_content_length": "487","http.scheme": "HTTP","http.status_code": 500,"http.target": "\u002fsession","http.user_agent": "selenium\u002f4.6.0 (java windows)"}}
2023-01-09 00:40:22 
2023-01-09 00:40:22 23:40:22.313 WARN [SeleniumSpanExporter$1.lambda$export$3] - {"traceId": "5a3052c7d98f9ba4cb40265f98fc5db6","eventTime": 1673221222317207900,"eventName": "HTTP request execution complete","attributes": {"http.flavor": 1,"http.handler_class": "org.openqa.selenium.grid.sessionqueue.local.LocalNewSessionQueue","http.host": "localhost:4444","http.method": "POST","http.request_content_length": "487","http.scheme": "HTTP","http.status_code": 500,"http.target": "\u002fsession","http.user_agent": "selenium\u002f4.6.0 (java windows)"}}
2023-01-09 00:40:22 
2023-01-09 00:40:22 23:40:22.261 WARN [SeleniumSpanExporter$1.lambda$export$3] - {"traceId": "84ac98f3cbb65e082edb9d008eb52d8a","eventTime": 1673221222264895800,"eventName": "HTTP request execution complete","attributes": {"http.flavor": 1,"http.handler_class": "org.openqa.selenium.grid.sessionqueue.local.LocalNewSessionQueue","http.host": "localhost:4444","http.method": "POST","http.request_content_length": "503","http.scheme": "HTTP","http.status_code": 500,"http.target": "\u002fsession","http.user_agent": "selenium\u002f4.6.0 (java windows)"}}
2023-01-09 00:40:22 

node

2023-01-09 00:34:29 2023-01-08 23:34:29,788 INFO Included extra file "/etc/supervisor/conf.d/selenium-grid-docker.conf" during parsing
2023-01-09 00:34:29 2023-01-08 23:34:29,806 INFO RPC interface 'supervisor' initialized
2023-01-09 00:34:29 2023-01-08 23:34:29,806 CRIT Server 'unix_http_server' running without any HTTP authentication checking
2023-01-09 00:34:29 2023-01-08 23:34:29,807 INFO supervisord started with pid 8
2023-01-09 00:34:30 2023-01-08 23:34:30,811 INFO spawned: 'socat' with pid 10
2023-01-09 00:34:30 2023-01-08 23:34:30,815 INFO spawned: 'selenium-grid-docker' with pid 11
2023-01-09 00:34:30 2023-01-08 23:34:30,840 INFO success: socat entered RUNNING state, process has stayed up for > than 0 seconds (startsecs)
2023-01-09 00:34:30 2023-01-08 23:34:30,841 INFO success: selenium-grid-docker entered RUNNING state, process has stayed up for > than 0 seconds (startsecs)
2023-01-09 00:34:30 2023-01-08 23:34:30,842 INFO exited: socat (exit status 0; expected)
2023-01-09 00:34:30 Starting Selenium Grid Node Docker...
2023-01-09 00:34:30 Tracing is disabled
2023-01-09 00:34:32 23:34:32.359 INFO [LoggingOptions.configureLogEncoding] - Using the system default encoding
2023-01-09 00:34:32 23:34:32.391 INFO [OpenTelemetryTracer.createTracer] - Using OpenTelemetry for tracing
2023-01-09 00:34:33 23:34:33.053 INFO [UnboundZmqEventBus.<init>] - Connecting to tcp://selenium-hub:4442 and tcp://selenium-hub:4443
2023-01-09 00:34:33 23:34:33.308 INFO [UnboundZmqEventBus.<init>] - Sockets created
2023-01-09 00:34:34 23:34:34.314 INFO [UnboundZmqEventBus.<init>] - Event bus ready
2023-01-09 00:34:34 23:34:34.762 INFO [NodeServer.createHandlers] - Reporting self as: http://172.24.0.3:5555
2023-01-09 00:34:34 23:34:34.815 INFO [NodeOptions.getSessionFactories] - Detected 4 available processors
2023-01-09 00:34:35 23:34:35.921 INFO [V141Docker.isContainerPresent] - Checking if container is present: dbd87025aeca
2023-01-09 00:34:36 23:34:36.038 INFO [Docker.getImage] - Obtaining image: selenium/standalone-chrome:4.7.2-20221219
2023-01-09 00:34:36 23:34:36.039 INFO [Docker.getImage] - Obtaining image: selenium/standalone-firefox:4.7.2-20221219
2023-01-09 00:34:36 23:34:36.036 INFO [Docker.getImage] - Obtaining image: selenium/standalone-edge:4.7.2-20221219
2023-01-09 00:34:36 23:34:36.046 INFO [V141Docker.getImage] - Listing local images: Reference{domain='docker.io', name='selenium/standalone-edge', tag='4.7.2-20221219', digest='null'}
2023-01-09 00:34:36 23:34:36.046 INFO [V141Docker.getImage] - Listing local images: Reference{domain='docker.io', name='selenium/standalone-firefox', tag='4.7.2-20221219', digest='null'}
2023-01-09 00:34:36 23:34:36.046 INFO [V141Docker.getImage] - Listing local images: Reference{domain='docker.io', name='selenium/standalone-chrome', tag='4.7.2-20221219', digest='null'}
2023-01-09 00:34:36 23:34:36.150 INFO [Docker.getImage] - Obtaining image: selenium/video:ffmpeg-4.3.1-20221219
2023-01-09 00:34:36 23:34:36.151 INFO [V141Docker.getImage] - Listing local images: Reference{domain='docker.io', name='selenium/video', tag='ffmpeg-4.3.1-20221219', digest='null'}
2023-01-09 00:34:36 23:34:36.191 INFO [Docker.getImage] - Obtaining image: selenium/video:ffmpeg-4.3.1-20221219
2023-01-09 00:34:36 23:34:36.191 INFO [V141Docker.getImage] - Listing local images: Reference{domain='docker.io', name='selenium/video', tag='ffmpeg-4.3.1-20221219', digest='null'}
2023-01-09 00:34:36 23:34:36.236 INFO [Docker.getImage] - Obtaining image: selenium/standalone-edge:4.7.2-20221219
2023-01-09 00:34:36 23:34:36.237 INFO [V141Docker.getImage] - Listing local images: Reference{domain='docker.io', name='selenium/standalone-edge', tag='4.7.2-20221219', digest='null'}
2023-01-09 00:34:36 23:34:36.289 INFO [DockerOptions.lambda$getDockerSessionFactories$1] - Mapping Capabilities {browserName: MicrosoftEdge} to docker image selenium/standalone-edge:4.7.2-20221219 4 times
2023-01-09 00:34:36 23:34:36.290 INFO [Docker.getImage] - Obtaining image: selenium/standalone-chrome:4.7.2-20221219
2023-01-09 00:34:36 23:34:36.291 INFO [V141Docker.getImage] - Listing local images: Reference{domain='docker.io', name='selenium/standalone-chrome', tag='4.7.2-20221219', digest='null'}
2023-01-09 00:34:36 23:34:36.338 INFO [DockerOptions.lambda$getDockerSessionFactories$1] - Mapping Capabilities {browserName: chrome} to docker image selenium/standalone-chrome:4.7.2-20221219 4 times
2023-01-09 00:34:36 23:34:36.339 INFO [Docker.getImage] - Obtaining image: selenium/standalone-firefox:4.7.2-20221219
2023-01-09 00:34:36 23:34:36.343 INFO [V141Docker.getImage] - Listing local images: Reference{domain='docker.io', name='selenium/standalone-firefox', tag='4.7.2-20221219', digest='null'}
2023-01-09 00:34:36 23:34:36.382 INFO [DockerOptions.lambda$getDockerSessionFactories$1] - Mapping Capabilities {browserName: firefox} to docker image selenium/standalone-firefox:4.7.2-20221219 4 times
2023-01-09 00:34:36 23:34:36.482 INFO [Node.<init>] - Binding additional locator mechanisms: relative, name, id
2023-01-09 00:34:36 23:34:36.965 INFO [NodeServer$1.start] - Starting registration process for Node http://172.24.0.3:5555
2023-01-09 00:34:36 23:34:36.968 INFO [NodeServer.execute] - Started Selenium node 4.7.2 (revision 4d4020c3b7): http://172.24.0.3:5555
2023-01-09 00:34:37 23:34:37.024 INFO [NodeServer$1.lambda$start$1] - Sending registration event...
2023-01-09 00:34:38 23:34:38.254 INFO [NodeServer.lambda$createHandlers$2] - Node has been added

The error logs in terminal look like:

[ERROR]   BaseTestNGRunnerTest.runScenario » SessionNotCreated Could not start a new session. Possible causes are invalid address of the remote server or browser start-up failure. 
Host info: host: 'DESKTOP-4JJ8P4R', ip: '192.168.50.152'
Build info: version: '4.6.0', revision: '79f1c02ae20'
System info: os.name: 'Windows 10', os.arch: 'amd64', os.version: '10.0', java.version: '17.0.5'
Driver info: org.openqa.selenium.remote.RemoteWebDriver
Command: [null, newSession {capabilities=[Capabilities {browserName: MicrosoftEdge, ms:edgeOptions: {args: [], extensions: []}, platformName: LINUX, se:recordVideo: true}], desiredCapabilities=Capabilities {browserName: MicrosoftEdge, ms:edgeOptions: {args: [], exte
nsions: []}, platformName: LINUX, se:recordVideo: true}}]
Capabilities {}

This is my docker-compose-v3-dynamic-grid.yml

version: "3"
services:
  node-docker:
    image: selenium/node-docker:4.7.2-20221219
    volumes:
      - ./assets/test-recordings:/opt/selenium/assets
      - ./src/test/resources/framework/selenium-grid-config/dynamic-grid-config.toml:/opt/bin/config.toml
    depends_on:
      - selenium-hub
    environment:
      - SE_EVENT_BUS_HOST=selenium-hub
      - SE_EVENT_BUS_PUBLISH_PORT=4442
      - SE_EVENT_BUS_SUBSCRIBE_PORT=4443

  selenium-hub:
    image: selenium/hub:4.7.2-20221219
    container_name: selenium-hub
    ports:
      - "4442:4442"
      - "4443:4443"
      - "4444:4444"

and this is my dynamic-grid-config.toml

[docker]
# Configs have a mapping between the Docker image to use and the capabilities that need to be matched to
# start a container with the given image.
configs = [
    "selenium/standalone-firefox:4.7.2-20221219", "{\"browserName\": \"firefox\"}",
    "selenium/standalone-chrome:4.7.2-20221219", "{\"browserName\": \"chrome\"}",
    "selenium/standalone-edge:4.7.2-20221219", "{\"browserName\": \"MicrosoftEdge\"}"
]

# URL for connecting to the docker daemon
# Most simple approach, leave it as http://127.0.0.1:2375, and mount /var/run/docker.sock.
# 127.0.0.1 is used because interally the container uses socat when /var/run/docker.sock is mounted
# If var/run/docker.sock is not mounted:
# Windows: make sure Docker Desktop exposes the daemon via tcp, and use http://host.docker.internal:2375.
# macOS: install socat and run the following command, socat -4 TCP-LISTEN:2375,fork UNIX-CONNECT:/var/run/docker.sock,
# then use http://host.docker.internal:2375.
# Linux: varies from machine to machine, please mount /var/run/docker.sock. If this does not work, please create an issue.
url = "http://host.docker.internal:2375"
# Docker image used for video recording
video-image = "selenium/video:ffmpeg-4.3.1-20221219"

# Uncomment the following section if you are running the node on a separate VM
# Fill out the placeholders with appropriate values
#[server]
#host = <ip-from-node-machine>
#port = <port-from-node-machine>

This is how I am initializing the remote webdriver for each of the browsers e.g. for MicrosoftEdge

EdgeOptions options = new EdgeOptions();
options.setCapability("platformName", platform);
options.setCapability("se:recordVideo", true);
driver = new RemoteWebDriver(new URL("http://localhost:4444/"), options);

The Grid and Node is setup but the docker does not creates new containers based on session requests dynamically.

dynamic-grid-overview

@pujagani
Copy link
Contributor

pujagani commented Jan 9, 2023

Based on the logs, the content length is not zero, which is the root cause of this PR is trying to solve. Please file an issue or reach out on https://www.selenium.dev/support/#ChatRoom to get further help.

@muodov
Copy link

muodov commented Jan 14, 2023

@pujagani What's the best way to be notified about the next release?

@titusfortner
Copy link
Member

We release it - https://github.com/SeleniumHQ/selenium/releases (you can set Github notifications to watch for just releases)
We blog it - https://www.selenium.dev/blog/
We tweet it - https://twitter.com/seleniumhq

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[🐛 Bug]: Dynamic Grid setup fails when the docker images are missing locally
9 participants