Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

[馃悰 Bug]: Grid 4.2.2 - unable to create new sessions #1605

Closed
lszasz opened this issue Jun 16, 2022 · 18 comments
Closed

[馃悰 Bug]: Grid 4.2.2 - unable to create new sessions #1605

lszasz opened this issue Jun 16, 2022 · 18 comments

Comments

@lszasz
Copy link

lszasz commented Jun 16, 2022

What happened?

Steps to reproduce:

  1. Have a session created and wait for the session to be removed due to inactivity
  2. Try to find element on the removed session for multiple times

Actual: The hub crashes and from this point on will be unable to create new session

You can reproduce the issue using the test from https://github.com/lszasz/selenium-test

Moreover /status is not reflecting that there are problems with the grid.

Command used to start Selenium Grid with Docker

version: '3.3'
services:
   seleniumhub:
       image: selenium/hub
       shm_size: 2g
       container_name: seleniumhub
       ports:
             - "4442:4442"
             - "4443:4443"
             - "4444:4444"
       environment:
             - TZ=Europe/Bucharest
             - SE_HUB_SE_OPTS="--log-level FINE"
             - SE_SESSION_REQUEST_TIMEOUT=1800
       volumes:
             - /dev/shm:/dev/shm
       deploy:
            resources:
                limits:
                  memory: 1G
                  cpus: '1'
   chrome:
       image: selenium/node-chrome
       shm_size: 2g
       depends_on:
             - seleniumhub
       volumes:
             - /tmp/automation:/home/seluser/Downloads
             - /dev/shm:/dev/shm

       environment:
        - AUTOHEAL_CONTAINER_LABEL=all
        - SCREEN_WIDTH=1920
        - SCREEN_HEIGHT=1080
        - TZ=Europe/Bucharest
        - SE_EVENT_BUS_HOST=seleniumhub
        - SE_EVENT_BUS_PUBLISH_PORT=4442
        - SE_EVENT_BUS_SUBSCRIBE_PORT=4443
        - SE_NODE_MAX_SESSIONS=1
        - SE_NODE_GRID_URL=http://clj-lc-tstdck01:4444
       deploy:
            resources:
                limits:
                  memory: 1G
                  cpus: '1'


I use the following commands to start the grid:
docker-compose scale seleniumhub=1
docker-compose scale chrome=10

Relevant log output

Session is deleted:

seleniumhub     | 13:03:37.692 INFO [GridModel.release] - Releasing slot for session id b12420f9f4126c5a3ab5e26170a0d7a0
seleniumhub     | 13:03:37.692 INFO [LocalSessionMap.lambda$new$0] - Deleted session from local Session Map, Id: b12420f9f4126c5a3ab5e26170a0d7a0
seleniumhub     | 13:03:55.820 WARN [SeleniumSpanExporter$1.lambda$export$1] - Unable to execute request for an existing session: Unable to find session with ID: b12420f9f4126c5a3ab5e26170a0d7a0
seleniumhub     | Build info: version: '4.2.2', revision: '683ccb65d6'
seleniumhub     | System info: host: 'acb1a4017c97', ip: '172.21.0.2', os.name: 'Linux', os.arch: 'amd64', os.version: '3.10.0-957.el7.x86_64', java.version: '11.0.15'
seleniumhub     | Driver info: driver.version: unknown
seleniumhub     | 13:03:55.821 WARN [SeleniumSpanExporter$1.lambda$export$1] - org.openqa.selenium.NoSuchSessionException: Unable to find session with ID: b12420f9f4126c5a3ab5e26170a0d7a0
seleniumhub     | Build info: version: '4.2.2', revision: '683ccb65d6'
seleniumhub     | System info: host: 'acb1a4017c97', ip: '172.21.0.2', os.name: 'Linux', os.arch: 'amd64', os.version: '3.10.0-957.el7.x86_64', java.version: '11.0.15'
seleniumhub     | Driver info: driver.version: unknown
seleniumhub     |       at org.openqa.selenium.grid.sessionmap.local.LocalSessionMap.get(LocalSessionMap.java:129)
seleniumhub     |       at org.openqa.selenium.grid.router.HandleSession.lambda$loadSessionId$4(HandleSession.java:159)
seleniumhub     |       at io.opentelemetry.context.Context.lambda$wrap$2(Context.java:224)
seleniumhub     |       at org.openqa.selenium.grid.router.HandleSession.execute(HandleSession.java:122)
seleniumhub     |       at org.openqa.selenium.remote.http.Route$PredicatedRoute.handle(Route.java:373)
seleniumhub     |       at org.openqa.selenium.remote.http.Route.execute(Route.java:68)
seleniumhub     |       at org.openqa.selenium.remote.http.Route$CombinedRoute.handle(Route.java:336)
seleniumhub     |       at org.openqa.selenium.remote.http.Route.execute(Route.java:68)
seleniumhub     |       at org.openqa.selenium.grid.router.Router.execute(Router.java:91)
seleniumhub     |       at org.openqa.selenium.grid.web.EnsureSpecCompliantResponseHeaders.lambda$apply$0(EnsureSpecCompliantResponseHeaders.java:34)
seleniumhub     |       at org.openqa.selenium.remote.http.Filter$1.execute(Filter.java:64)
seleniumhub     |       at org.openqa.selenium.remote.http.Route$CombinedRoute.handle(Route.java:336)
seleniumhub     |       at org.openqa.selenium.remote.http.Route.execute(Route.java:68)
seleniumhub     |       at org.openqa.selenium.remote.http.Route$NestedRoute.handle(Route.java:270)
seleniumhub     |       at org.openqa.selenium.remote.http.Route.execute(Route.java:68)
seleniumhub     |       at org.openqa.selenium.remote.http.Route$CombinedRoute.handle(Route.java:336)
seleniumhub     |       at org.openqa.selenium.remote.http.Route.execute(Route.java:68)
seleniumhub     |       at org.openqa.selenium.remote.http.Route$CombinedRoute.handle(Route.java:336)
seleniumhub     |       at org.openqa.selenium.remote.http.Route.execute(Route.java:68)
seleniumhub     |       at org.openqa.selenium.remote.AddWebDriverSpecHeaders.lambda$apply$0(AddWebDriverSpecHeaders.java:35)
seleniumhub     |       at org.openqa.selenium.remote.ErrorFilter.lambda$apply$0(ErrorFilter.java:44)
seleniumhub     |       at org.openqa.selenium.remote.http.Filter$1.execute(Filter.java:64)
seleniumhub     |       at org.openqa.selenium.remote.ErrorFilter.lambda$apply$0(ErrorFilter.java:44)
seleniumhub     |       at org.openqa.selenium.remote.http.Filter$1.execute(Filter.java:64)
seleniumhub     |       at org.openqa.selenium.netty.server.SeleniumHandler.lambda$channelRead0$0(SeleniumHandler.java:44)
seleniumhub     |       at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
seleniumhub     |       at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
seleniumhub     |       at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
seleniumhub     |       at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
seleniumhub     |       at java.base/java.lang.Thread.run(Thread.java:829)



Everything else after will fail:
Client side error:
org.openqa.selenium.SessionNotCreatedException: Could not start a new session. Possible causes are invalid address of the remote server or browser start-up failure.
Build info: version: '4.2.2', revision: '683ccb65d6'
System info: host: 'CLJSDELSZASZ01', ip: '192.168.1.193', os.name: 'Windows 10', os.arch: 'amd64', os.version: '10.0', java.version: '11'
Driver info: org.openqa.selenium.remote.RemoteWebDriver
Command: [null, newSession {capabilities=[Capabilities {browserName: chrome, goog:chromeOptions: {args: [], extensions: []}}], desiredCapabilities=Capabilities {browserName: chrome, goog:chromeOptions: {args: [], extensions: []}}}]
Capabilities {}

Hub error:
seleniumhub     | 13:06:20.298 WARN [SpanWrappedHttpHandler.execute] - Unable to execute request: Expected to read a START_MAP but instead have: END. Last 0 characters read:
seleniumhub     | Build info: version: '4.2.2', revision: '683ccb65d6'
seleniumhub     | System info: host: 'acb1a4017c97', ip: '172.21.0.2', os.name: 'Linux', os.arch: 'amd64', os.version: '3.10.0-957.el7.x86_64', java.version: '11.0.15'
seleniumhub     | Driver info: driver.version: unknown
seleniumhub     | org.openqa.selenium.json.JsonException: Expected to read a START_MAP but instead have: END. Last 0 characters read:
seleniumhub     | Build info: version: '4.2.2', revision: '683ccb65d6'
seleniumhub     | System info: host: 'acb1a4017c97', ip: '172.21.0.2', os.name: 'Linux', os.arch: 'amd64', os.version: '3.10.0-957.el7.x86_64', java.version: '11.0.15'
seleniumhub     | Driver info: driver.version: unknown
seleniumhub     |       at org.openqa.selenium.json.JsonInput.expect(JsonInput.java:307)
seleniumhub     |       at org.openqa.selenium.json.JsonInput.beginObject(JsonInput.java:229)
seleniumhub     |       at org.openqa.selenium.remote.NewSessionPayload.getOss(NewSessionPayload.java:306)
seleniumhub     |       at org.openqa.selenium.remote.NewSessionPayload.<init>(NewSessionPayload.java:84)
seleniumhub     |       at org.openqa.selenium.remote.NewSessionPayload.create(NewSessionPayload.java:118)
seleniumhub     |       at org.openqa.selenium.grid.data.SessionRequest.<init>(SessionRequest.java:65)
seleniumhub     |       at org.openqa.selenium.grid.sessionqueue.NewSessionQueue.lambda$new$0(NewSessionQueue.java:66)
seleniumhub     |       at org.openqa.selenium.remote.http.Route$TemplatizedRoute.handle(Route.java:192)
seleniumhub     |       at org.openqa.selenium.remote.http.Route.execute(Route.java:68)
seleniumhub     |       at org.openqa.selenium.remote.http.Route$CombinedRoute.handle(Route.java:336)
seleniumhub     |       at org.openqa.selenium.remote.http.Route.execute(Route.java:68)
seleniumhub     |       at org.openqa.selenium.grid.sessionqueue.NewSessionQueue.execute(NewSessionQueue.java:120)
seleniumhub     |       at org.openqa.selenium.remote.tracing.SpanWrappedHttpHandler.execute(SpanWrappedHttpHandler.java:86)
seleniumhub     |       at org.openqa.selenium.remote.http.Filter$1.execute(Filter.java:64)
seleniumhub     |       at org.openqa.selenium.remote.http.Route$CombinedRoute.handle(Route.java:336)
seleniumhub     |       at org.openqa.selenium.remote.http.Route.execute(Route.java:68)
seleniumhub     |       at org.openqa.selenium.grid.router.Router.execute(Router.java:91)
seleniumhub     |       at org.openqa.selenium.grid.web.EnsureSpecCompliantResponseHeaders.lambda$apply$0(EnsureSpecCompliantResponseHeaders.java:34)
seleniumhub     |       at org.openqa.selenium.remote.http.Filter$1.execute(Filter.java:64)
seleniumhub     |       at org.openqa.selenium.remote.http.Route$CombinedRoute.handle(Route.java:336)
seleniumhub     |       at org.openqa.selenium.remote.http.Route.execute(Route.java:68)
seleniumhub     |       at org.openqa.selenium.remote.http.Route$NestedRoute.handle(Route.java:270)
seleniumhub     |       at org.openqa.selenium.remote.http.Route.execute(Route.java:68)
seleniumhub     |       at org.openqa.selenium.remote.http.Route$CombinedRoute.handle(Route.java:336)
seleniumhub     |       at org.openqa.selenium.remote.http.Route.execute(Route.java:68)
seleniumhub     |       at org.openqa.selenium.remote.http.Route$CombinedRoute.handle(Route.java:336)
seleniumhub     |       at org.openqa.selenium.remote.http.Route.execute(Route.java:68)
seleniumhub     |       at org.openqa.selenium.remote.AddWebDriverSpecHeaders.lambda$apply$0(AddWebDriverSpecHeaders.java:35)
seleniumhub     |       at org.openqa.selenium.remote.ErrorFilter.lambda$apply$0(ErrorFilter.java:44)
seleniumhub     |       at org.openqa.selenium.remote.http.Filter$1.execute(Filter.java:64)
seleniumhub     |       at org.openqa.selenium.remote.ErrorFilter.lambda$apply$0(ErrorFilter.java:44)
seleniumhub     |       at org.openqa.selenium.remote.http.Filter$1.execute(Filter.java:64)
seleniumhub     |       at org.openqa.selenium.netty.server.SeleniumHandler.lambda$channelRead0$0(SeleniumHandler.java:44)
seleniumhub     |       at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
seleniumhub     |       at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
seleniumhub     |       at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
seleniumhub     |       at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
seleniumhub     |       at java.base/java.lang.Thread.run(Thread.java:829)

Operating System

Centos 7

Docker Selenium version (tag)

selenium/hub:4.2.2

@lszasz lszasz changed the title [馃悰 Bug]: Grid 4.2.2 is unable to create new sessions [馃悰 Bug]: Grid 4.2.2 - unable to create new sessions Jun 16, 2022
@namilkimfree

This comment was marked as spam.

@diemol
Copy link
Member

diemol commented Jun 20, 2022

Thank you for reporting. I am able to reproduce, however I do not get the exact same stacktrace but the Grid is not usable after the NoSuchSessionException is triggered.

Not sure how to solve this for now, as the initial server implementation has a few assumptions that make it hard to fix. I need to re-think it and see how we can fix this.

@diemol diemol mentioned this issue Jun 27, 2022
8 tasks
@FerrisYu
Copy link

Hitting same issue in our test environment with max 60 sessions in our selenium hub. Any suggestion on recent stable version of the hub I could upgrade. So far we failed two attempt to upgrade to Selenium4 and sticking to Selenium3 due to session issues we had in 4.1.3 and 4.3.0. Thanks

@tcongthang
Copy link

tcongthang commented Jul 24, 2022

Got the same error, seem the session is released by grid after 5 minutes inactive. happens with 4.3 latest version also

selenium_v4_docker_grid-node-docker-1 | 14:55:07.923 INFO [Container.stop] - Stopping container 506d4645a7016e38b31cb588235732750cd8d0dbfeedce8298e8c7216217f704
selenium_v4_docker_grid-node-docker-1 | 14:55:12.621 INFO [SessionSlot.stop] - Stopping session 081b1d9e13a96a2708a20608ffe63f06
selenium-hub | 14:55:12.629 INFO [GridModel.release] - Releasing slot for session id 081b1d9e13a96a2708a20608ffe63f06
selenium-hub | 14:55:12.628 INFO [LocalSessionMap.lambda$new$0] - Deleted session from local Session Map, Id: 081b1d9e13a96a2708a20608ffe63f06
selenium-hub | 14:55:38.204 WARN [SeleniumSpanExporter$1.lambda$export$1] - Unable to execute request for an existing session: Unable to find session with ID: 081b1d9e13a96a2708a20608ffe63f06
selenium-hub | Build info: version: '4.3.0', revision: 'a4995e2c09*'
selenium-hub | System info: host: 'f9feb90f4553', ip: '172.22.0.2', os.name: 'Linux', os.arch: 'amd64', os.version: '5.15.0-41-generic', java.version: '11.0.15'
selenium-hub | Driver info: driver.version: unknown
selenium-hub | 14:55:38.206 WARN [SeleniumSpanExporter$1.lambda$export$1] - org.openqa.selenium.NoSuchSessionException: Unable to find session with ID: 081b1d9e13a96a2708a20608ffe63f06

@SACTAN
Copy link

SACTAN commented Jul 25, 2022

This issue is also there on selenium 4.3.0 version, We did selenium gird setup without docker and still issue is there with below error,
Driver info: driver.version: unknown
org.openqa.selenium.json.JsonException: Expected to read a START_MAP but instead have: END. Last 0 characters read:
Build info: version: '4.3.0', revision: 'a49935e2c109*'
System info: host: 'abcdeftestrf', ip: '100.189.167.991', os.name: 'Windows', os.arch: 'amd64', os.version: '10.0', java.version: '1.8.0_271'
Driver info: driver.version: unknown
at org.openqa.selenium.json.JsonInput.expect(JsonInput.java:307)
at org.openqa.selenium.json.JsonInput.beginObject(JsonInput.java:229)
at org.openqa.selenium.remote.NewSessionPayload.getOss(NewSessionPayload.java:306)
at org.openqa.selenium.remote.NewSessionPayload.(NewSessionPayload.java:84)
at org.openqa.selenium.remote.NewSessionPayload.create(NewSessionPayload.java:118)
at org.openqa.selenium.grid.data.SessionRequest.(SessionRequest.java:65)
at org.openqa.selenium.grid.sessionqueue.NewSessionQueue.lambda$new$0(NewSessionQueue.java:66)
at org.openqa.selenium.remote.http.Route$TemplatizedRoute.handle(Route.java:192)
at org.openqa.selenium.remote.http.Route.execute(Route.java:68)
at org.openqa.selenium.remote.http.Route$CombinedRoute.handle(Route.java:336)
at org.openqa.selenium.remote.http.Route.execute(Route.java:68)
at org.openqa.selenium.grid.sessionqueue.NewSessionQueue.execute(NewSessionQueue.java:120)
at org.openqa.selenium.remote.tracing.SpanWrappedHttpHandler.execute(SpanWrappedHttpHandler.java:86)
at org.openqa.selenium.remote.http.Filter$1.execute(Filter.java:64)
at org.openqa.selenium.remote.http.Route$CombinedRoute.handle(Route.java:336)
at org.openqa.selenium.remote.http.Route.execute(Route.java:68)
at org.openqa.selenium.grid.router.Router.execute(Router.java:91)
at org.openqa.selenium.grid.web.CheckOriginHeader.lambda$apply$0(CheckOriginHeader.java:66)
at org.openqa.selenium.grid.web.CheckContentTypeHeader.lambda$apply$0(CheckContentTypeHeader.java:70)
at org.openqa.selenium.grid.web.EnsureSpecCompliantResponseHeaders.lambda$apply$0(EnsureSpecCompliantResponseHeaders.java:34)
at org.openqa.selenium.remote.http.Filter$1.execute(Filter.java:64)
at org.openqa.selenium.remote.http.Route$CombinedRoute.handle(Route.java:336)
at org.openqa.selenium.remote.http.Route.execute(Route.java:68)
at org.openqa.selenium.remote.http.Route$NestedRoute.handle(Route.java:270)
at org.openqa.selenium.remote.http.Route.execute(Route.java:68)
at org.openqa.selenium.remote.http.Route$CombinedRoute.handle(Route.java:336)
at org.openqa.selenium.remote.http.Route.execute(Route.java:68)
at org.openqa.selenium.remote.http.Route$CombinedRoute.handle(Route.java:336)
at org.openqa.selenium.remote.http.Route.execute(Route.java:68)
at org.openqa.selenium.remote.AddWebDriverSpecHeaders.lambda$apply$0(AddWebDriverSpecHeaders.java:35)
at org.openqa.selenium.remote.ErrorFilter.lambda$apply$0(ErrorFilter.java:44)
at org.openqa.selenium.remote.http.Filter$1.execute(Filter.java:64)
at org.openqa.selenium.remote.ErrorFilter.lambda$apply$0(ErrorFilter.java:44)
at org.openqa.selenium.remote.http.Filter$1.execute(Filter.java:64)
at org.openqa.selenium.netty.server.SeleniumHandler.lambda$channelRead0$0(SeleniumHandler.java:44)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)

@tcongthang
Copy link

tcongthang commented Aug 2, 2022

@diemol
We already set SE_NODE_SESSION_TIMEOUT=1800 but don't know why grid release/delete sessions which inactive 300s at this log

Releasing slot for session id b12420f9f4126c5a3ab5e26170a0d7a0 seleniumhub | 13:03:37.692 INFO [LocalSessionMap.lambda$new$0] - Deleted session from local Session Map

@roderiano
Copy link

I'm having the same issue with only one session running interactions with the session ended. Does anyone know a solution? The grid version used to run the scripts is 4.5.0. Below is the grid log and script to reproduce the bug.

Grid Log

20:34:12.907 INFO [Hub.execute] - Started Selenium Hub 4.5.0 (revision fe167b119a): http://172.18.0.2:4444
20:34:14.805 INFO [Node.<init>] - Binding additional locator mechanisms: name, id, relative
20:34:15.058 INFO [GridModel.setAvailability] - Switching Node 13ba1304-3bfb-4210-9c5c-17e01a3b31ff (uri: http://172.18.0.3:5555) from DOWN to UP
20:34:15.058 INFO [LocalDistributor.add] - Added node 13ba1304-3bfb-4210-9c5c-17e01a3b31ff at http://172.18.0.3:5555. Health check every 120s
20:34:27.585 INFO [LocalDistributor.newSession] - Session request received by the Distributor: 
 [Capabilities {browserName: chrome, goog:chromeOptions: {args: [--kiosk-printing, --no-sandbox, --disable-dev-shm-usage], extensions: [], prefs: {download.default_directory: /home/seluser/Downloads}}, pageLoadStrategy: normal}]
20:34:28.185 INFO [LocalDistributor.newSession] - Session created by the Distributor. Id: 1b14713841bb35cdcb0863772c314a76 
 Caps: Capabilities {acceptInsecureCerts: false, browserName: chrome, browserVersion: 105.0.5195.52, chrome: {chromedriverVersion: 105.0.5195.52 (412c95e51883..., userDataDir: /tmp/.org.chromium.Chromium...}, goog:chromeOptions: {debuggerAddress: localhost:41641}, networkConnectionEnabled: false, pageLoadStrategy: normal, platformName: LINUX, proxy: {}, se:cdp: ws://172.18.0.3:4444/sessio..., se:cdpVersion: 105.0.5195.52, se:vnc: ws://172.18.0.3:4444/sessio..., se:vncEnabled: true, se:vncLocalAddress: ws://172.18.0.3:7900, setWindowRect: true, strictFileInteractability: false, timeouts: {implicit: 0, pageLoad: 300000, script: 30000}, unhandledPromptBehavior: dismiss and notify, webauthn:extension:credBlob: true, webauthn:extension:largeBlob: true, webauthn:virtualAuthenticators: true}
20:39:44.906 INFO [LocalSessionMap.lambda$new$0] - Deleted session from local Session Map, Id: 1b14713841bb35cdcb0863772c314a76
20:39:44.907 INFO [GridModel.release] - Releasing slot for session id 1b14713841bb35cdcb0863772c314a76
20:40:20.211 WARN [SeleniumSpanExporter$1.lambda$export$1] - Unable to execute request for an existing session: Unable to find session with ID: 1b14713841bb35cdcb0863772c314a76
Build info: version: '4.5.0', revision: 'fe167b119a'
System info: os.name: 'Linux', os.arch: 'aarch64', os.version: '5.10.124-linuxkit', java.version: '11.0.16'
Driver info: driver.version: unknown
20:40:20.212 WARN [SeleniumSpanExporter$1.lambda$export$1] - org.openqa.selenium.NoSuchSessionException: Unable to find session with ID: 1b14713841bb35cdcb0863772c314a76
Build info: version: '4.5.0', revision: 'fe167b119a'
System info: os.name: 'Linux', os.arch: 'aarch64', os.version: '5.10.124-linuxkit', java.version: '11.0.16'
Driver info: driver.version: unknown
        at org.openqa.selenium.grid.sessionmap.local.LocalSessionMap.get(LocalSessionMap.java:129)
        at org.openqa.selenium.grid.router.HandleSession.lambda$loadSessionId$4(HandleSession.java:159)
        at io.opentelemetry.context.Context.lambda$wrap$2(Context.java:224)
        at org.openqa.selenium.grid.router.HandleSession.execute(HandleSession.java:122)
        at org.openqa.selenium.remote.http.Route$PredicatedRoute.handle(Route.java:373)
        at org.openqa.selenium.remote.http.Route.execute(Route.java:68)
        at org.openqa.selenium.remote.http.Route$CombinedRoute.handle(Route.java:336)
        at org.openqa.selenium.remote.http.Route.execute(Route.java:68)
        at org.openqa.selenium.grid.router.Router.execute(Router.java:91)
        at org.openqa.selenium.grid.web.EnsureSpecCompliantResponseHeaders.lambda$apply$0(EnsureSpecCompliantResponseHeaders.java:34)
        at org.openqa.selenium.remote.http.Filter$1.execute(Filter.java:64)
        at org.openqa.selenium.remote.http.Route$CombinedRoute.handle(Route.java:336)
        at org.openqa.selenium.remote.http.Route.execute(Route.java:68)
        at org.openqa.selenium.remote.http.Route$NestedRoute.handle(Route.java:270)
        at org.openqa.selenium.remote.http.Route.execute(Route.java:68)
        at org.openqa.selenium.remote.http.Route$CombinedRoute.handle(Route.java:336)
        at org.openqa.selenium.remote.http.Route.execute(Route.java:68)
        at org.openqa.selenium.remote.http.Route$CombinedRoute.handle(Route.java:336)
        at org.openqa.selenium.remote.http.Route.execute(Route.java:68)
        at org.openqa.selenium.remote.AddWebDriverSpecHeaders.lambda$apply$0(AddWebDriverSpecHeaders.java:35)
        at org.openqa.selenium.remote.ErrorFilter.lambda$apply$0(ErrorFilter.java:44)
        at org.openqa.selenium.remote.http.Filter$1.execute(Filter.java:64)
        at org.openqa.selenium.remote.ErrorFilter.lambda$apply$0(ErrorFilter.java:44)
        at org.openqa.selenium.remote.http.Filter$1.execute(Filter.java:64)
        at org.openqa.selenium.netty.server.SeleniumHandler.lambda$channelRead0$0(SeleniumHandler.java:44)
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:829)

20:40:20.214 WARN [SeleniumSpanExporter$1.lambda$export$3] - {"traceId": "bb1b1689ef4f58008cb4b9a5b0566e46","eventTime": 1666384820208521208,"eventName": "exception","attributes": {"exception.message": "Unable to execute request for an existing session: Unable to find session with ID: 1b14713841bb35cdcb0863772c314a76\nBuild info: version: '4.5.0', revision: 'fe167b119a'\nSystem info: os.name: 'Linux', os.arch: 'aarch64', os.version: '5.10.124-linuxkit', java.version: '11.0.16'\nDriver info: driver.version: unknown","exception.stacktrace": "org.openqa.selenium.NoSuchSessionException: Unable to find session with ID: 1b14713841bb35cdcb0863772c314a76\nBuild info: version: '4.5.0', revision: 'fe167b119a'\nSystem info: os.name: 'Linux', os.arch: 'aarch64', os.version: '5.10.124-linuxkit', java.version: '11.0.16'\nDriver info: driver.version: unknown\n\tat org.openqa.selenium.grid.sessionmap.local.LocalSessionMap.get(LocalSessionMap.java:129)\n\tat org.openqa.selenium.grid.router.HandleSession.lambda$loadSessionId$4(HandleSession.java:159)\n\tat io.opentelemetry.context.Context.lambda$wrap$2(Context.java:224)\n\tat org.openqa.selenium.grid.router.HandleSession.execute(HandleSession.java:122)\n\tat org.openqa.selenium.remote.http.Route$PredicatedRoute.handle(Route.java:373)\n\tat org.openqa.selenium.remote.http.Route.execute(Route.java:68)\n\tat org.openqa.selenium.remote.http.Route$CombinedRoute.handle(Route.java:336)\n\tat org.openqa.selenium.remote.http.Route.execute(Route.java:68)\n\tat org.openqa.selenium.grid.router.Router.execute(Router.java:91)\n\tat org.openqa.selenium.grid.web.EnsureSpecCompliantResponseHeaders.lambda$apply$0(EnsureSpecCompliantResponseHeaders.java:34)\n\tat org.openqa.selenium.remote.http.Filter$1.execute(Filter.java:64)\n\tat org.openqa.selenium.remote.http.Route$CombinedRoute.handle(Route.java:336)\n\tat org.openqa.selenium.remote.http.Route.execute(Route.java:68)\n\tat org.openqa.selenium.remote.http.Route$NestedRoute.handle(Route.java:270)\n\tat org.openqa.selenium.remote.http.Route.execute(Route.java:68)\n\tat org.openqa.selenium.remote.http.Route$CombinedRoute.handle(Route.java:336)\n\tat org.openqa.selenium.remote.http.Route.execute(Route.java:68)\n\tat org.openqa.selenium.remote.http.Route$CombinedRoute.handle(Route.java:336)\n\tat org.openqa.selenium.remote.http.Route.execute(Route.java:68)\n\tat org.openqa.selenium.remote.AddWebDriverSpecHeaders.lambda$apply$0(AddWebDriverSpecHeaders.java:35)\n\tat org.openqa.selenium.remote.ErrorFilter.lambda$apply$0(ErrorFilter.java:44)\n\tat org.openqa.selenium.remote.http.Filter$1.execute(Filter.java:64)\n\tat org.openqa.selenium.remote.ErrorFilter.lambda$apply$0(ErrorFilter.java:44)\n\tat org.openqa.selenium.remote.http.Filter$1.execute(Filter.java:64)\n\tat org.openqa.selenium.netty.server.SeleniumHandler.lambda$channelRead0$0(SeleniumHandler.java:44)\n\tat java.base\u002fjava.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)\n\tat java.base\u002fjava.util.concurrent.FutureTask.run(FutureTask.java:264)\n\tat java.base\u002fjava.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)\n\tat java.base\u002fjava.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)\n\tat java.base\u002fjava.lang.Thread.run(Thread.java:829)\n","exception.type": "org.openqa.selenium.NoSuchSessionException","http.flavor": 1,"http.handler_class": "org.openqa.selenium.grid.router.HandleSession","http.host": "127.0.0.1:4444","http.method": "POST","http.request_content_length": "1191","http.scheme": "HTTP","http.target": "\u002fsession\u002f1b14713841bb35cdcb0863772c314a76\u002felement","http.user_agent": "selenium\u002f4.5.0 (python mac)","session.id": "1b14713841bb35cdcb0863772c314a76"}}

Python Script

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.common.exceptions import InvalidSessionIdException
import time					 
					 
configuration = {"browserName": "chrome"}
options = webdriver.ChromeOptions()
options.add_argument("--kiosk-printing")
options.add_argument("--no-sandbox")
options.add_argument("--disable-dev-shm-usage")

driver = webdriver.Remote(
    command_executor="http://127.0.0.1:4444/wd/hub",
    desired_capabilities=configuration,
    options=options,
)
try:
    driver.maximize_window() # Note: driver.maximize_window does not work on Linux selenium version v2, instead set window size and window position like driver.set_window_position(0,0) and driver.set_window_size(1920,1080)
    driver.get("https://github.com/lszasz/selenium-test")
    time.sleep(320)
    for i in range(0, 5):
        try:
            driver.find_element(By.XPATH, "//*[contains(concat(' ', @class, ' '), ' x-component ') and contains(concat(' ', @class, ' '), ' x-fit-item ') and not(contains(@class, 'x-hidden-offsets'))]//*[contains(concat(' ', @class, ' '), ' x-panel ') and not(contains(@class, 'x-hidden-offsets')) and count(.//*[contains(concat(' ', @class, ' '), ' x-toolbar ') and count(.//h3[contains(concat(' ', @class, ' '), ' x-toolbar-item ') and contains(text(),'Bwb_simple_trapr25164550427.txt')]) > 0]) > 0]//*[contains(concat(' ', @class, ' '), ' x-tab-bar ') and count(.//a[contains(concat(' ', @class, ' '), ' x-tab-active ') and count(*//text()[.='Details']) > 0]) > 0]//following-sibling::*[contains(concat(' ', @class, ' '), ' x-panel-body ')]/*[contains(concat(' ', @class, ' '), ' x-tabpanel-child ') and not(contains(@class, 'x-hidden-offsets'))]//*[contains(concat(' ', @class, ' '), ' x-toolbar ')]//a[contains(concat(' ', @class, ' '), ' x-btn ') and (count(*//text()[contains(.,'Download Package')]) > 0 or count(*//text()[contains(.,'Download Source File')]) > 0 or count(*//text()[contains(.,'Download SDLXLIFF')]) > 0) and count(.//*[contains(concat(' ', @class, ' '), ' fa-download  ')]) > 0]").click()
        except Exception as e:
            if type(e) == InvalidSessionIdException:
                break
            print(e)
finally:
    driver.quit()

@krmahadevan
Copy link
Contributor

@diemol - There seems to be two issues that are mentioned in this defect.

  • After a timeout causes a session to be cleaned up at the Grid level, the Grid fails to honour any new further new sessions. Cannot reproduce in 4.5.3

Here's the docker compose file I used

version: "3"
services:
  chrome:
    image: selenium/node-chrome:4.5.3-20221024
    shm_size: 2gb
    depends_on:
      - selenium-hub
    environment:
      - SE_NODE_SESSION_TIMEOUT=2
      - SE_EVENT_BUS_HOST=selenium-hub
      - SE_EVENT_BUS_PUBLISH_PORT=4442
      - SE_EVENT_BUS_SUBSCRIBE_PORT=4443
  selenium-hub:
    image: selenium/hub:4.5.3-20221024
    container_name: selenium-hub
    ports:
      - "4442:4442"
      - "4443:4443"
      - "4444:4444"

Here's a sample that I ran

import java.net.URL;
import java.util.concurrent.TimeUnit;
import org.openqa.selenium.chrome.ChromeOptions;
import org.openqa.selenium.remote.RemoteWebDriver;

public class SessionsExample {

  public static void main(String[] args) throws Exception {
    try {
      System.err.println("Running First test with an induced timeout");
      runTest(true);
    } catch (Exception ignored) {
    }
    System.err.println("Running Second test without any timeouts");
    runTest(false);
  }

  private static void runTest(boolean addTimeOut) throws Exception {
    RemoteWebDriver driver = null;
    try {
      driver = new RemoteWebDriver(new URL("http://localhost:4444"), new ChromeOptions());
      System.err.println("Session ID " + driver.getSessionId().toString());
      driver.get("https://www.selenium.dev/");
      if (addTimeOut) {
        TimeUnit.SECONDS.sleep(45);
        System.err.println("Slept for 45 seconds");
      }
      System.err.println("Title " + driver.getTitle());
    } finally {
      if (driver != null) {
        driver.quit();
      }
    }
  }
}

So if the first issue is no longer happening (atleast not for me on 4.5.3) then maybe we can close this off.

@lszasz @roderiano @SACTAN - Thoughts ?

@diemol
Copy link
Member

diemol commented Oct 29, 2022

This was an actual issue caused by the retries in the findElement method. When the session timed out, the client tried to do a findElement and the Grid replied with a 404 right away, but this did not allow the Grid to read the whole content from the request. This leaves an input pipe waiting for content and eventually causing a lock. That is what causes the Grid to hang and no process new requests (like serving GraphQL requests from the Grid UI).

The fix closes the input pipeline when a 404 is responded. It will be part of 4.6.0, which should come out in the next few days.

joshmgrant pushed a commit to joshmgrant/selenium that referenced this issue Oct 29, 2022
In some cases the resource is not found, and if we have not read completely the input
stream, a read/write lock will happen since the client gets a response and won't stream
the remaining content. Closing the pipe avoids a lock and leaves the RequestConverter
ready for upcoming requests.

Fixes SeleniumHQ/docker-selenium#1646
Fixes SeleniumHQ/docker-selenium#1605
@manolinux
Copy link

manolinux commented Nov 3, 2022

This was an actual issue caused by the retries in the findElement method. When the session timed out, the client tried to do a findElement and the Grid replied with a 404 right away, but this did not allow the Grid to read the whole content from the request. This leaves an input pipe waiting for content and eventually causing a lock. That is what causes the Grid to hang and no process new requests (like serving GraphQL requests from the Grid UI).

The fix closes the input pipeline when a 404 is responded. It will be part of 4.6.0, which should come out in the next few days.

Same happening to our Team. After a some hours, doing some selenium work, closing windows and quitting sessions, we face exactly this. But session seems to be created, but operating on it (setting a page_load_timeout) raises a 404, and then all is the same. Also happens with other operations, it's not necessarily linked to set_page_load_timeout. It seems the session is created but in a buggy way, and server cannot later on make use of it.

@lszasz
Copy link
Author

lszasz commented Dec 1, 2022

Updated finally from 3.x a few days ago to 4.6.0 and everything is running smooth. Thanks a lot @diemol!

@lior277
Copy link

lior277 commented Jan 2, 2023

This was an actual issue caused by the retries in the findElement method. When the session timed out, the client tried to do a findElement and the Grid replied with a 404 right away, but this did not allow the Grid to read the whole content from the request. This leaves an input pipe waiting for content and eventually causing a lock. That is what causes the Grid to hang and no process new requests (like serving GraphQL requests from the Grid UI).

The fix closes the input pipeline when a 404 is responded. It will be part of 4.6.0, which should come out in the next few days.

i am on 4.7 and it steel happening.
element is, Exception: Timed out after 30 seconds
----> OpenQA.Selenium.WebDriverException : invalid session id
TearDown : OpenQA.Selenium.WebDriverException : invalid session id
and the strange thing is that this exception always happening on the same method

@krmahadevan
Copy link
Contributor

@lior277 - Can you please add some additional context?

  1. Are you running against the Selenium Grid ? If yes what version ?
  2. Can you please help share a sample test case, that can be executed to reproduce the problem ?

@lior277
Copy link

lior277 commented Jan 4, 2023

@lior277 - Can you please add some additional context?

  1. Are you running against the Selenium Grid ? If yes what version ?
  2. Can you please help share a sample test case, that can be executed to reproduce the problem ?
  1. Yes and the image version is 4.7.2-20221219
  2. I cant share a test because it happens in different places

@diemol
Copy link
Member

diemol commented Jan 4, 2023

If you are facing this issue, please create a new issue with a complete way to reproduce it, otherwise we cannot really have a look at it.

@patilsha
Copy link

patilsha commented Mar 8, 2023

@diemol @krmahadevan I'm also facing this issue with latest version of selenium Grid 4.8.1 (revision 8ebccac989)
It was coming in Selenium Grid 4.1.1 version but even after upgrading it to latest, I'm still finding the same issue.
Is there any open ticket for this issue or any upcoming fixes in new release? Thanks

@wieta
Copy link

wieta commented Mar 9, 2023

Im using a container (standalone-chrome latest ver.) and facing the same issue. Downgraded to 110.0-chromedriver-110.0-grid-4.8.0-20230210 and everything works fine.

Copy link

github-actions bot commented Dec 9, 2023

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

@github-actions github-actions bot locked and limited conversation to collaborators Dec 9, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests