Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[🐛 Bug]: Deleted session from local Session Map #1922

Closed
madhankumar866 opened this issue Aug 23, 2023 · 22 comments
Closed

[🐛 Bug]: Deleted session from local Session Map #1922

madhankumar866 opened this issue Aug 23, 2023 · 22 comments
Labels
I-question

Comments

@madhankumar866
Copy link

What happened?

Session is deleted from local Session Map when process is running state.

Command used to start Selenium Grid with Docker

Helm chart version: 0.19.0

Relevant log output

Driver info: driver.version: unknown                                                                                                                                                                                            │
│ 05:28:34.801 WARN [SeleniumSpanExporter$1.lambda$export$1] - org.openqa.selenium.NoSuchSessionException: Unable to find session with ID: 4c78101fb6f55af089b0c9235190f24c                                                       │
│ Build info: version: '4.10.0', revision: 'c14d967899'                                                                                                                                                                           │
│ System info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.10.184-175.731.amzn2.x86_64', java.version: '11.0.19'                                                                                                           │
│ Driver info: driver.version: unknown                                                                                                                                                                                            │
│ Build info: version: '4.10.0', revision: 'c14d967899'                                                                                                                                                                           │
│ System info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.10.184-175.731.amzn2.x86_64', java.version: '11.0.19'                                                                                                           │
│ Driver info: driver.version: unknown                                                                                                                                                                                            │
│     at jdk.internal.reflect.GeneratedConstructorAccessor15.newInstance(Unknown Source)                                                                                                                                          │
│     at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)                                                                                                  │
│     at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490)                                                                                                                                                │
│     at org.openqa.selenium.remote.ErrorCodec.decode(ErrorCodec.java:144)                                                                                                                                                        │
│     at org.openqa.selenium.grid.web.Values.get(Values.java:47)                                                                                                                                                                  │
│     at org.openqa.selenium.grid.sessionmap.remote.RemoteSessionMap.makeRequest(RemoteSessionMap.java:118)                                                                                                                       │
│     at org.openqa.selenium.grid.sessionmap.remote.RemoteSessionMap.get(RemoteSessionMap.java:89)                                                                                                                                │
│     at org.openqa.selenium.grid.router.HandleSession.lambda$loadSessionId$4(HandleSession.java:172)                                                                                                                             │
│     at io.opentelemetry.context.Context.lambda$wrap$2(Context.java:224)                                                                                                                                                         │
│     at org.openqa.selenium.grid.router.HandleSession.execute(HandleSession.java:125)                                                                                                                                            │
│     at org.openqa.selenium.remote.http.Route$PredicatedRoute.handle(Route.java:384)                                                                                                                                             │
│     at org.openqa.selenium.remote.http.Route.execute(Route.java:69)                                                                                                                                                             │
│     at org.openqa.selenium.remote.http.Route$CombinedRoute.handle(Route.java:347)                                                                                                                                               │
│     at org.openqa.selenium.remote.http.Route.execute(Route.java:69)                                                                                                                                                             │
│     at org.openqa.selenium.grid.router.Router.execute(Router.java:87)                                                                                                                                                           │
│     at org.openqa.selenium.grid.web.EnsureSpecCompliantResponseHeaders.lambda$apply$0(EnsureSpecCompliantResponseHeaders.java:34)                                                                                               │
│     at org.openqa.selenium.remote.http.Filter$1.execute(Filter.java:63)                                                                                                                                                         │
│     at org.openqa.selenium.remote.http.Route$CombinedRoute.handle(Route.java:347)                                                                                                                                               │
│     at org.openqa.selenium.remote.http.Route.execute(Route.java:69)                                                                                                                                                             │
│     at org.openqa.selenium.remote.http.Route$CombinedRoute.handle(Route.java:347)                                                                                                                                               │
│     at org.openqa.selenium.remote.http.Route.execute(Route.java:69)                                                                                                                                                             │
│     at org.openqa.selenium.remote.http.Route$CombinedRoute.handle(Route.java:347)                                                                                                                                               │
│     at org.openqa.selenium.remote.http.Route.execute(Route.java:69)                                                                                                                                                             │
│     at org.openqa.selenium.remote.http.Route$CombinedRoute.handle(Route.java:347)                                                                                                                                               │
│     at org.openqa.selenium.remote.http.Route.execute(Route.java:69)                                                                                                                                                             │
│     at org.openqa.selenium.remote.AddWebDriverSpecHeaders.lambda$apply$0(AddWebDriverSpecHeaders.java:35)                                                                                                                       │
│     at org.openqa.selenium.remote.ErrorFilter.lambda$apply$0(ErrorFilter.java:44)                                                                                                                                               │
│     at org.openqa.selenium.remote.http.Filter$1.execute(Filter.java:63)                                                                                                                                                         │
│     at org.openqa.selenium.remote.ErrorFilter.lambda$apply$0(ErrorFilter.java:44)    
     at org.openqa.selenium.remote.ErrorFilter.lambda$apply$0(ErrorFilter.java:44)                                                                                                                                               │
│     at org.openqa.selenium.remote.http.Filter$1.execute(Filter.java:63)                                                                                                                                                         │
│     at org.openqa.selenium.netty.server.SeleniumHandler.lambda$channelRead0$0(SeleniumHandler.java:44)                                                                                                                          │
│     at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)                                                                                                                                        │
│     at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)                                                                                                                                                       │
│     at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)                                                                                                                                │
│     at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)                                                                                                                                │
│     at java.base/java.lang.Thread.run(Thread.java:829)                                                                                                                                                                          │
│                                                                                                                                                                                                                                 │
│ 05:28:34.802 WARN [SeleniumSpanExporter$1.lambda$export$3] - {"traceId": "5ff2c81f10fb90e41ca8ec140e97622a","eventTime": 1692768514800795089,"eventName": "exception","attributes": {"exception.message": "Unable to execute re │
│                                                                                                                                                                                                                                 │
│ 05:28:34.815 WARN [SeleniumSpanExporter$1.lambda$export$1] - Unable to execute request for an existing session: Unable to find session with ID: 4c78101fb6f55af089b0c9235190f24c                                                │
│ Build info: version: '4.10.0', revision: 'c14d967899'                                                                                                                                                                           │
│ System info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.10.184-175.731.amzn2.x86_64', java.version: '11.0.19'                                                                                                           │
│ Driver info: driver.version: unknown                                                                                                                                                                                            │
│ Build info: version: '4.10.0', revision: 'c14d967899'                                                                                                                                                                           │
│ System info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.10.184-175.731.amzn2.x86_64', java.version: '11.0.19'                                                                                                           │
│ Driver info: driver.version: unknown                                                                                                                                                                                            │
│ 05:28:34.816 WARN [SeleniumSpanExporter$1.lambda$export$1] - org.openqa.selenium.NoSuchSessionException: Unable to find session with ID: 4c78101fb6f55af089b0c9235190f24c                                                       │
│ Build info: version: '4.10.0', revision: 'c14d967899'                                                                                                                                                                           │
│ System info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.10.184-175.731.amzn2.x86_64', java.version: '11.0.19'                                                                                                           │
│ Driver info: driver.version: unknown                                                                                                                                                                                            │
│ Build info: version: '4.10.0', revision: 'c14d967899'                                                                                                                                                                           │
│ System info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.10.184-175.731.amzn2.x86_64', java.version: '11.0.19'                                                                                                           │
│ Driver info: driver.version: unknown                                                                                                                                                                                            │
│     at jdk.internal.reflect.GeneratedConstructorAccessor15.newInstance(Unknown Source)                                                                                                                                          │
│     at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)                                                                                                  │
│     at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490)                                                                                                                                                │
│     at org.openqa.selenium.remote.ErrorCodec.decode(ErrorCodec.java:144)                                                                                                                                                        │
│     at org.openqa.selenium.grid.web.Values.get(Values.java:47)         
   at org.openqa.selenium.remote.ErrorCodec.decode(ErrorCodec.java:144)                                                                                                                                                        │
│     at org.openqa.selenium.grid.web.Values.get(Values.java:47)                                                                                                                                                                  │
│     at org.openqa.selenium.grid.sessionmap.remote.RemoteSessionMap.makeRequest(RemoteSessionMap.java:118)                                                                                                                       │
│     at org.openqa.selenium.grid.sessionmap.remote.RemoteSessionMap.get(RemoteSessionMap.java:89)                                                                                                                                │
│     at org.openqa.selenium.grid.router.HandleSession.lambda$loadSessionId$4(HandleSession.java:172)                                                                                                                             │
│     at io.opentelemetry.context.Context.lambda$wrap$2(Context.java:224)                                                                                                                                                         │
│     at org.openqa.selenium.grid.router.HandleSession.execute(HandleSession.java:125)                                                                                                                                            │
│     at org.openqa.selenium.remote.http.Route$PredicatedRoute.handle(Route.java:384)                                                                                                                                             │
│     at org.openqa.selenium.remote.http.Route.execute(Route.java:69)                                                                                                                                                             │
│     at org.openqa.selenium.remote.http.Route$CombinedRoute.handle(Route.java:347)                                                                                                                                               │
│     at org.openqa.selenium.remote.http.Route.execute(Route.java:69)                                                                                                                                                             │
│     at org.openqa.selenium.grid.router.Router.execute(Router.java:87)                                                                                                                                                           │
│     at org.openqa.selenium.grid.web.EnsureSpecCompliantResponseHeaders.lambda$apply$0(EnsureSpecCompliantResponseHeaders.java:34)                                                                                               │
│     at org.openqa.selenium.remote.http.Filter$1.execute(Filter.java:63)                                                                                                                                                         │
│     at org.openqa.selenium.remote.http.Route$CombinedRoute.handle(Route.java:347)                                                                                                                                               │
│     at org.openqa.selenium.remote.http.Route.execute(Route.java:69)                                                                                                                                                             │
│     at org.openqa.selenium.remote.http.Route$CombinedRoute.handle(Route.java:347)                                                                                                                                               │
│     at org.openqa.selenium.remote.http.Route.execute(Route.java:69)                                                                                                                                                             │
│     at org.openqa.selenium.remote.http.Route$CombinedRoute.handle(Route.java:347)                                                                                                                                               │
│     at org.openqa.selenium.remote.http.Route.execute(Route.java:69)                                                                                                                                                             │
│     at org.openqa.selenium.remote.http.Route$CombinedRoute.handle(Route.java:347)                                                                                                                                               │
│     at org.openqa.selenium.remote.http.Route.execute(Route.java:69)                                                                                                                                                             │
│     at org.openqa.selenium.remote.AddWebDriverSpecHeaders.lambda$apply$0(AddWebDriverSpecHeaders.java:35)                                                                                                                       │
│     at org.openqa.selenium.remote.ErrorFilter.lambda$apply$0(ErrorFilter.java:44)                                                                                                                                               │
│     at org.openqa.selenium.remote.http.Filter$1.execute(Filter.java:63)                                                                                                                                                         │
│     at org.openqa.selenium.remote.ErrorFilter.lambda$apply$0(ErrorFilter.java:44)                                                                                                                                               │
│     at org.openqa.selenium.remote.http.Filter$1.execute(Filter.java:63)                                                                                                                                                         │
│     at org.openqa.selenium.netty.server.SeleniumHandler.lambda$channelRead0$0(SeleniumHandler.java:44)                                                                                                                          │
│     at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)                                                                                                                                        │
│     at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)                                                                                                                                                       │
│     at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)                                                                                                                                │
│     at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)                                                                                                                                │
│     at java.base/java.lang.Thread.run(Thread.java:829)                                                                                                                                                                          │
│                                                                                                                                                                                                                                 │
│ 05:28:34.816 WARN [SeleniumSpanExporter$1.lambda$export$3] - {"traceId": "ad8ee3185de8a2138bb9b54d00128214","eventTime": 1692768514815048369,"eventName": "exception","attributes": {"exception.message": "Unable to execute re │
│                                                                                                                                                                                                                                 │
│

Operating System

amazon linux

Docker Selenium version (tag)

4.10.0-20230607
@github-actions
Copy link

@madhankumar866, thank you for creating this issue. We will troubleshoot it as soon as we can.

Info for maintainers

Triage this issue by using labels.

If information is missing, add a helpful comment and then I-issue-template label.

If the issue is a question, add the I-question label.

If the issue is valid but there is no time to troubleshoot it, consider adding the help wanted label.

If the issue requires changes or fixes from an external project (e.g., ChromeDriver, GeckoDriver, MSEdgeDriver, W3C), add the applicable G-* label, and it will provide the correct link and auto-close the issue.

After troubleshooting the issue, please add the R-awaiting answer label.

Thank you!

@diemol
Copy link
Member

diemol commented Aug 23, 2023

That is expected when the session idle timeout is reached

https://www.selenium.dev/documentation/grid/configuration/cli_options/

@diemol diemol closed this as not planned Won't fix, can't repro, duplicate, stale Aug 23, 2023
@madhankumar866
Copy link
Author

That is expected when the session idle timeout is reached

https://www.selenium.dev/documentation/grid/configuration/cli_options/

Yes You Are Correct
"Let X be the session-timeout in 300 seconds. The Node will automatically kill a session that has not had any activity in the last X seconds. This will release the slot for other tests"

But In My Case It happens when Activity is going on.

@madhankumar866
Copy link
Author

madhankumar866 commented Aug 23, 2023 •
edited

Getting session id deleted

event Deleted session from local Session Map  
logger org.openqa.selenium.grid.sessionmap.local.LocalSessionMap  
session.id 8111f43d2d4760fd33d2170887e4a332

But My timeout is 300seconds when it reach 5 minutes of running period it is getting timeout

@diemol
Copy link
Member

diemol commented Aug 23, 2023

What activity is going on when this happens?

@madhankumar866
Copy link
Author

Clicking Inbox in Email and suddenly getting session id not found exception

1
2

@diemol
Copy link
Member

diemol commented Aug 23, 2023

Was there any activity before that? Is the session is idle it ends.

@github-actions
Copy link

💬 Please ask questions at:

@madhankumar866
Copy link
Author

Was there any activity before that? Is the session is idle it ends.

No I am running it in 1 node per request.
i am just Testing using only one It is Stopping the session and remove the seesion id when process is on going.
i can get the activity using "jaeger" above Screen shot.

Can you suggest me how can i troubleshoot it. I am just using a simple script to check it.

@diemol
Copy link
Member

diemol commented Aug 23, 2023

https://github.com/SeleniumHQ/docker-selenium#grid-url-and-session-timeout

@madhankumar866
Copy link
Author

Deleted session from local Session Map

Tried That as well no response Session Termiated after 300 seconds
session

@diemol
Copy link
Member

diemol commented Aug 23, 2023

You are only sharing bits of your setup, if you need help troubleshooting you need to provide a way to reproduce this, with code scripts and instructions showing how to execute them.

@madhankumar866
Copy link
Author

You are only sharing bits of your setup, if you need help troubleshooting you need to provide a way to reproduce this, with code scripts and instructions showing how to execute them.

Infrastructure ruuning in Aws EKS
Grid is installaed using helm chart Version 0.19.0
Below are the Changes done in chart

isolateComponents: true, 

extraEnvironmentVariables:
	- name: SE_ENABLE_TRACING
	  value: "true"
	- name: JAVA_OPTS
	  value: "-Dotel.traces.exporter=jaeger -Dotel.exporter.jaeger.endpoint=http://jaegar-collector:14250 -Dotel.resource.attributes=service.name=selenium-hub"
	- name: SE_NODE_SESSION_TIMEOUT      
	  value: "600"


keda is enable, 
using only chrome browser

Python 3.11.2
Aws EKs Version 1.26

I Use a Jump host, Internal Network loadbalancer For Grid Communcation and script execution.

Below is my simple script, Session id is deleted after 5 minutes.

from distutils import config
import pymysql
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By
from time import sleep
from multiprocessing import Pool
import sys
import re
import os
import zipfile
from array import array
import datetime
import random
import requests
import pickle
import config
from selenium.webdriver.common.action_chains import ActionChains
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.ui import WebDriverWait
import queue
from concurrent.futures import ThreadPoolExecutor
from selenium.common.exceptions import InvalidSessionIdException

grid_url = 'aws.internal.loadbalancer'

chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument("ignore-certificate-errors")
chrome_options.add_argument("disable-infobars")
chrome_options.add_argument("--disable-popup-blocking")
chrome_options.add_argument("--disable-notifications")
chrome_options.add_argument('--disable-gpu')
desired_capabilities = DesiredCapabilities.CHROME.copy()
driver = webdriver.Remote(command_executor=grid_url,desired_capabilities=desired_capabilities ,options=webdriver.ChromeOptions())
sleep(3)
driver.execute_script('location.reload()')
sleep(5)
session_id=driver.session_id
print(session_id)
try:
driver.get('http://www.google.com')
sleep(10)
driver.get('http://www.yahoo.com')
st_time = datetime.datetime.now().strftime("%Y-%m-%d %H:%M:%S")
print('hhhhhhhh',st_time)
sleep(60)
ed_time = datetime.datetime.now().strftime("%Y-%m-%d %H:%M:%S")
print('vtttttt',ed_time)
driver.get('http://www.bing.com')
sleep(60)
driver.get('http://www.google.com')
sleep(60)
driver.get('http://www.yahoo.com')
sleep(60)
driver.get('http://www.bing.com')
sleep(60)
driver.get('http://www.google.com')
sleep(60)
driver.get('http://www.yahoo.com')
sleep(60)
driver.quit()
except:
pass

@diemol
Copy link
Member

diemol commented Aug 23, 2023

What about the container logs?

@madhankumar866
Copy link
Author

madhankumar866 commented Aug 24, 2023 •
edited

Here you can view jaeger portal for tracing

a68f2c074ad2d4390afcc9fcf06a0741-2078220606.us-east-1.elb.amazonaws.com

session.id=22a07972d1c9c0afa00d6ce2efa27349

This is Chrome Container logs Below

	2023-08-24 01:21:42,171 INFO Included extra file "/etc/supervisor/conf.d/selenium.conf" during parsing
	2023-08-24 01:21:42,174 INFO RPC interface 'supervisor' initialized
	2023-08-24 01:21:42,174 CRIT Server 'unix_http_server' running without any HTTP authentication checking
	2023-08-24 01:21:42,174 INFO supervisord started with pid 8
	2023-08-24 01:21:43,176 INFO spawned: 'xvfb' with pid 10
	2023-08-24 01:21:43,178 INFO spawned: 'vnc' with pid 11
	2023-08-24 01:21:43,180 INFO spawned: 'novnc' with pid 12
	2023-08-24 01:21:43,188 INFO spawned: 'selenium-node' with pid 14
	2023-08-24 01:21:43,216 INFO success: selenium-node entered RUNNING state, process has stayed up for > than 0 seconds (startsecs)
	Generating Selenium Config
	Configuring server...
	Setting up SE_NODE_HOST...
	Setting up SE_NODE_PORT...
	Tracing is disabled
	Selenium Grid Node configuration: 
	[events]
	publish = "tcp://selenium-event-bus:4442"
	subscribe = "tcp://selenium-event-bus:4443"

	[node]
	grid-url = "http://selenium-router.default:4444"
	session-timeout = "300"
	override-max-sessions = false
	detect-drivers = false
	drain-after-session-count = 0
	max-sessions = 1

	[[node.driver-configuration]]
	display-name = "chrome"
	stereotype = '{"browserName": "chrome", "browserVersion": "114.0", "platformName": "Linux"}'
	max-sessions = 1

	Starting Selenium Grid Node...
	2023-08-24 01:21:44,237 INFO success: xvfb entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
	2023-08-24 01:21:44,237 INFO success: vnc entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
	2023-08-24 01:21:44,237 INFO success: novnc entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
	01:21:44.915 INFO [LoggingOptions.configureLogEncoding] - Using the system default encoding
	01:21:44.923 INFO [OpenTelemetryTracer.createTracer] - Using OpenTelemetry for tracing
	01:21:45.323 INFO [UnboundZmqEventBus.<init>] - Connecting to tcp://selenium-event-bus:4442 and tcp://selenium-event-bus:4443
	01:21:45.461 INFO [UnboundZmqEventBus.<init>] - Sockets created
	01:21:46.520 INFO [UnboundZmqEventBus.<init>] - Event bus ready
	01:21:46.825 INFO [NodeServer.createHandlers] - Reporting self as: http://192.168.104.242:5555
	01:21:46.859 INFO [NodeOptions.getSessionFactories] - Detected 1 available processors
	01:21:47.036 INFO [NodeOptions.report] - Adding chrome for {"browserName": "chrome","browserVersion": "114.0","platformName": "linux","se:noVncPort": 7900,"se:vncEnabled": true} 1 times (Host)
	01:21:47.122 INFO [Node.<init>] - Binding additional locator mechanisms: relative
	01:21:47.564 INFO [NodeServer$1.start] - Starting registration process for Node http://192.168.104.242:5555
	01:21:47.615 INFO [NodeServer.execute] - Started Selenium node 4.10.0 (revision c14d967899): http://192.168.104.242:5555
	01:21:47.634 INFO [NodeServer$1.lambda$start$1] - Sending registration event...
	01:21:48.138 INFO [NodeServer.lambda$createHandlers$2] - Node has been added
	01:21:48.329 INFO [DriverService$Builder.getLogOutput] - Driver logs no longer sent to console by default; https://www.selenium.dev/documentation/webdriver/drivers/service/#setting-log-output
	01:21:50.161 INFO [LocalNode.newSession] - Session created by the Node. Id: 22a07972d1c9c0afa00d6ce2efa27349, Caps: Capabilities {acceptInsecureCerts: false, browserName: chrome, browserVersion: 114.0.5735.106, chrome: {chromedriverVersion: 114.0.5735.90 (386bc09e8f4f..., userDataDir: /tmp/.com.google.Chrome.RWKXPO}, goog:chromeOptions: {debuggerAddress: localhost:34959}, networkConnectionEnabled: false, pageLoadStrategy: normal, platformName: linux, proxy: Proxy(), se:bidiEnabled: false, se:cdp: ws://selenium-router.defaul..., se:cdpVersion: 114.0.5735.106, se:vnc: ws://selenium-router.defaul..., se:vncEnabled: true, se:vncLocalAddress: ws://192.168.104.242:7900, setWindowRect: true, strictFileInteractability: false, timeouts: {implicit: 0, pageLoad: 300000, script: 30000}, unhandledPromptBehavior: dismiss and notify, webauthn:extension:credBlob: true, webauthn:extension:largeBlob: true, webauthn:extension:minPinLength: true, webauthn:extension:prf: true, webauthn:virtualAuthenticators: true}
	01:24:52.951 INFO [ProxyNodeWebsockets.createWsEndPoint] - Establishing connection to ws://192.168.104.242:7900

@farioas
Copy link

farioas commented Sep 1, 2023 •
edited

Hi,

I am facing exactly the same problem after switching from static chrome-nodes pool to keda + node autoscaler.
I noticed that this problem exists only for the first 5-10 sessions, when ec2 vms are just created and pods that are registered as chrome nodes are running on them.

selenium-hub logs, see timestamps:

08:13:47.975 INFO [LocalDistributor.newSession] - Session created by the Distributor. Id: 94d543e2998857818a50ecb810679c93
...
08:15:52.260 INFO [GridModel.release] - Releasing slot for session id 94d543e2998857818a50ecb810679c93
08:15:52.260 INFO [LocalSessionMap.lambda$new$0] - Deleted session from local Session Map, Id: 94d543e2998857818a50ecb810679c93
...
08:15:52.399 WARN [SeleniumSpanExporter$1.lambda$export$1] - Unable to execute request for an existing session: Unable to find session with ID: 94d543e2998857818a50ecb810679c93
Build info: version: '4.11.0', revision: '040bc5406b'
System info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.10.178-162.673.amzn2.x86_64', java.version: '11.0.20'
Driver info: driver.version: unknown
08:15:52.399 WARN [SeleniumSpanExporter$1.lambda$export$1] - org.openqa.selenium.NoSuchSessionException: Unable to find session with ID: 94d543e2998857818a50ecb810679c93
Build info: version: '4.11.0', revision: '040bc5406b'
...
08:47:30.459 WARN [SeleniumSpanExporter$1.lambda$export$1] - Unable to execute request for an existing session: Unable to find session with ID: 94d543e2998857818a50ecb810679c93
08:47:30.459 WARN [SeleniumSpanExporter$1.lambda$export$1] - org.openqa.selenium.NoSuchSessionException: Unable to find session with ID: 94d543e2998857818a50ecb810679c93

The session 94d543e2998857818a50ecb810679c93 was deleted after 123 seconds, then it was requested and then it was requested again after 32 minutes.

values:

hub:
  extraEnvironmentVariables:
    - name: SE_SESSION_REQUEST_TIMEOUT
      value: "1800"
    - name: SE_SESSION_RETRY_INTERVAL
      value: "20"
...
chromeNode:
  extraEnvironmentVariables:
    - name: SE_NODE_SESSION_TIMEOUT
      value: "300"

@farioas
Copy link

farioas commented Sep 4, 2023

Why is it closed? @diemol

@diemol
Copy link
Member

diemol commented Sep 4, 2023

Because no one has provided a way to reproduce it and the original poster did not reply anymore.

We also need help to maintain that Keda integration. Seems the issue lies with Keda, or a better understanding of the configuration.

I am considering moving the chart out of this repository because we have no time to maintain it. Hopefully, someone will jump in.

@farioas
Copy link

farioas commented Sep 4, 2023

I don't think the problem is in KEDA, since it's responsible only for a node provision.

@diemol
Copy link
Member

diemol commented Sep 4, 2023

I think it also shutsdown nodes, but again, without a way to reproduce and someone who can help have a look we cannot do much.

@farioas
Copy link

farioas commented Sep 4, 2023

Well, I have no time digging in it, but I resolved it by switching from ScaledJob to ScaledObject with chrome-node as a target. Also, I've added terminationGracePeriod for Deployment:

autoscaling:
  enableWithExistingKEDA: true
  scalingType: deployment

chromeNode:
  terminationGracePeriodSeconds: 3600

I suppose the problem is somewhere in DRAIN_AFTER_SESSION_COUNT=1 for ScaledJob, it deletes the session from Session Map before it was completed.

@github-actions GitHub Actions
Copy link

github-actions bot commented Dec 9, 2023

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

@github-actions github-actions bot locked and limited conversation to collaborators Dec 9, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
I-question
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@farioas @diemol @madhankumar866