PEBCAK: Ops-er needs more instructions

[tsluyter]

It seems that I, without any Dev background, need a little bit more documentation on how to actually run the tests in an unattended fashion.

Assume that I fire up the Selenium Grid (confirmed that the UI is available on :4444, with three browsers waiting for instructions). How would I then get the Grid to perform tests against either the locally running JuiceShop, or a remotely hosted on?

If I strip testng.xml down to the Selenium Grid tests, I do see jobs queued on the SGrid UI, but the browsers themselves don't appear to be getting the assignments. Instead, there's a three minute timeout, after which the queued jobs go away again.

Logs from mvn test

[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO] Running TestSuite
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
Aug 01, 2022 5:52:46 AM org.openqa.selenium.remote.tracing.opentelemetry.OpenTelemetryTracer createTracer
INFO: Using OpenTelemetry for tracing


[ERROR] Tests run: 19, Failures: 3, Errors: 0, Skipped: 16, Time elapsed: 184.097 s <<< FAILURE! - in TestSuite
[ERROR] io.github.mfaisalkhatri.tests.theinternet.FormAuthenticationTests.setupTest[remote-chrome](0)  Time elapsed: 183.981 s  <<< FAILURE!
org.openqa.selenium.SessionNotCreatedException: 
Could not start a new session. Possible causes are invalid address of the remote server or browser start-up failure.
Build info: version: '4.3.0', revision: 'a4995e2c09*'
System info: host: 'devsecops', ip: '127.0.2.1', os.name: 'Linux', os.arch: 'amd64', os.version: '5.4.0-110-generic', java.version: '17.0.3'
Driver info: org.openqa.selenium.remote.RemoteWebDriver
Command: [null, newSession {capabilities=[Capabilities {browserName: chrome, browserVersion: 101}], desiredCapabilities=Capabilities {browserName: chrome, browserVersion: 101}}]
Capabilities {}
	at org.openqa.selenium.remote.RemoteWebDriver.execute(RemoteWebDriver.java:587)
	at org.openqa.selenium.remote.RemoteWebDriver.startSession(RemoteWebDriver.java:264)
	at org.openqa.selenium.remote.RemoteWebDriver.<init>(RemoteWebDriver.java:179)
	at org.openqa.selenium.remote.RemoteWebDriver.<init>(RemoteWebDriver.java:161)
	at io.github.mfaisalkhatri.drivers.DriverManager.createDriver(DriverManager.java:75)
	at io.github.mfaisalkhatri.tests.base.BaseSuiteSetup.setupTest(BaseSuiteSetup.java:18)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at org.testng.internal.invokers.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:139)
	at org.testng.internal.invokers.MethodInvocationHelper.invokeMethodConsideringTimeout(MethodInvocationHelper.java:69)
	at org.testng.internal.invokers.ConfigInvoker.invokeConfigurationMethod(ConfigInvoker.java:361)
	at org.testng.internal.invokers.ConfigInvoker.invokeConfigurations(ConfigInvoker.java:296)
	at org.testng.internal.invokers.TestMethodWorker.invokeBeforeClassMethods(TestMethodWorker.java:180)
	at org.testng.internal.invokers.TestMethodWorker.run(TestMethodWorker.java:122)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
	at org.testng.TestRunner.privateRun(TestRunner.java:806)
	at org.testng.TestRunner.run(TestRunner.java:601)
	at org.testng.SuiteRunner.runTest(SuiteRunner.java:433)
	at org.testng.SuiteRunner$SuiteWorker.run(SuiteRunner.java:471)
	at org.testng.internal.thread.ThreadUtil.lambda$execute$0(ThreadUtil.java:58)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
	at java.base/java.lang.Thread.run(Thread.java:833)
Caused by: java.lang.RuntimeException: NettyHttpHandler request execution error
	at org.openqa.selenium.remote.http.netty.NettyHttpHandler.makeCall(NettyHttpHandler.java:76)
	at org.openqa.selenium.remote.http.AddSeleniumUserAgent.lambda$apply$0(AddSeleniumUserAgent.java:42)
	at org.openqa.selenium.remote.http.Filter.lambda$andFinally$1(Filter.java:56)
	at org.openqa.selenium.remote.http.netty.NettyHttpHandler.execute(NettyHttpHandler.java:49)
	at org.openqa.selenium.remote.http.AddSeleniumUserAgent.lambda$apply$0(AddSeleniumUserAgent.java:42)
	at org.openqa.selenium.remote.http.Filter.lambda$andFinally$1(Filter.java:56)
	at org.openqa.selenium.remote.http.netty.NettyClient.execute(NettyClient.java:98)
	at org.openqa.selenium.remote.tracing.TracedHttpClient.execute(TracedHttpClient.java:55)
	at org.openqa.selenium.remote.ProtocolHandshake.createSession(ProtocolHandshake.java:120)
	at org.openqa.selenium.remote.ProtocolHandshake.createSession(ProtocolHandshake.java:102)
	at org.openqa.selenium.remote.ProtocolHandshake.createSession(ProtocolHandshake.java:67)
	at org.openqa.selenium.remote.HttpCommandExecutor.execute(HttpCommandExecutor.java:156)
	at org.openqa.selenium.remote.TracedCommandExecutor.execute(TracedCommandExecutor.java:51)
	at org.openqa.selenium.remote.RemoteWebDriver.execute(RemoteWebDriver.java:569)
	... 25 more
Caused by: java.util.concurrent.ExecutionException: java.util.concurrent.TimeoutException: Request timeout to selenium-hub/127.0.0.1:4444 after 180000 ms
	at java.base/java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:396)
	at java.base/java.util.concurrent.CompletableFuture.get(CompletableFuture.java:2096)
	at org.asynchttpclient.netty.NettyResponseFuture.get(NettyResponseFuture.java:206)
	at org.openqa.selenium.remote.http.netty.NettyHttpHandler.makeCall(NettyHttpHandler.java:59)
	... 38 more
Caused by: java.util.concurrent.TimeoutException: Request timeout to selenium-hub/127.0.0.1:4444 after 180000 ms
	at org.asynchttpclient.netty.timeout.TimeoutTimerTask.expire(TimeoutTimerTask.java:43)
	at org.asynchttpclient.netty.timeout.RequestTimeoutTimerTask.run(RequestTimeoutTimerTask.java:50)
	at io.netty.util.HashedWheelTimer$HashedWheelTimeout.run(HashedWheelTimer.java:715)
	at io.netty.util.concurrent.ImmediateExecutor.execute(ImmediateExecutor.java:34)
	at io.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:703)
	at io.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:790)
	at io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:503)
	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
	... 1 more

Logs from selenium-hub, after three minutes

selenium-hub    | 05:57:48.503 WARN [SeleniumSpanExporter$1.lambda$export$0] - {"traceId": "2d117effc641c9831e481049220d5e01","eventTime": 1659333468502501009,"eventName": "HTTP request execution complete","attributes": {"http.flavor": 1,"http.handler_class": "org.openqa.selenium.grid.sessionqueue.local.LocalNewSessionQueue","http.host": "selenium-hub:4444","http.method": "POST","http.request_content_length": "223","http.scheme": "HTTP","http.status_code": 500,"http.target": "\u002fsession","http.user_agent": "selenium\u002f4.3.0 (java unix)"}}
selenium-hub    | 
selenium-hub    | 05:57:48.511 WARN [SeleniumSpanExporter$1.lambda$export$0] - {"traceId": "611ee885f3e4c4e5d737154faf598a1b","eventTime": 1659333468509915752,"eventName": "HTTP request execution complete","attributes": {"http.flavor": 1,"http.handler_class": "org.openqa.selenium.grid.sessionqueue.local.LocalNewSessionQueue","http.host": "selenium-hub:4444","http.method": "POST","http.request_content_length": "223","http.scheme": "HTTP","http.status_code": 500,"http.target": "\u002fsession","http.user_agent": "selenium\u002f4.3.0 (java unix)"}}
selenium-hub    | 
selenium-hub    | 05:57:48.497 WARN [SeleniumSpanExporter$1.lambda$export$0] - {"traceId": "62ac86eb1e36b5b3f68cd7458f94281a","eventTime": 1659333468496574636,"eventName": "HTTP request execution complete","attributes": {"http.flavor": 1,"http.handler_class": "org.openqa.selenium.grid.sessionqueue.local.LocalNewSessionQueue","http.host": "selenium-hub:4444","http.method": "POST","http.request_content_length": "223","http.scheme": "HTTP","http.status_code": 500,"http.target": "\u002fsession","http.user_agent": "selenium\u002f4.3.0 (java unix)"}}

Setup:

• Vagrant box, for Ubuntu 20.04
• Docker and Docker-compose, also via Ubuntu repo
• OpenJDK 17, via Ubuntu repo
• Maven 3.8.6, via Apache.org

Original message: Removed. I was severely misunderstanding things.

[tsluyter]

EDIT: Removed, top post has current, proper details.

[tsluyter]

I've gotten the local JuiceShop tests to work! To do this, I had to make a few changes:

• In JuiceShopTests.java, change websiteLink to "http://juice-shop:3000/#/".
• In DriverManager.java, remove the caps.setVersion() calls.
• In docker-compose-v3.yml, JuiceShop port needs to be 3000, not 3001.
• In docker-compose-v3.yml, I removed all the specific version tags for the Selenium containers.

For my personal purposes, I have changed testng.xml so it would only run the Selenium Grid-based tests for JuiceShop.

Now mvn test works and only gives a failure on one test (Firefox, make payment)

[mfaisalkhatri]

@tsluyter Thanks a lot for starting this conversation and raising ticket, I have made a lot of changes as per your suggestion in the code base and have also updated the Readme with all the steps to run the tests in local. Also have updated the medium-blog by adding the steps to run the tests. Appreciate your feedback.

[tsluyter]

Thank you for all your hard work! I'll go read the blog post and the README.md again.

[tsluyter]

The README is awesome, thank you! And so's the update to your blog post. Thank you!