Explore ATH run against containerized selenium #556

olivergondza · 2020-01-28T12:40:36Z

Permit ATH to utilize selenium containers to host browser, driver and selenium server. This si tangential to what ath-container do, but it integrates nicely with buildPlugin.

This is intended to be a way to use ATH from plugins source code.

Downstream PRs:

olivergondza · 2020-02-05T07:56:00Z

@jenkinsci/code-reviewers && @jglick, please have a look. Mainly at the documentation part: https://github.com/olivergondza/acceptance-test-harness/blob/prototype-containerized-selenium/docs/EXTERNAL.md (note the example repositories do not have the change merged jet)

amuniz

Very nice job @olivergondza!!

amuniz · 2020-02-05T08:36:08Z

src/main/java/org/jenkinsci/test/acceptance/FallbackConfig.java

+                    return "Kill and remove selenium container";
+                }
+            };
+            Thread.sleep(3000); // Give the container and selenium some time to spawn


This seems a bit fragile. Isn't there a way to check if the container fully started? Maybe checking some logs entries?

Another way is to poll for a 200 response for up to some time. That will save some time if it's faster than 3000

jglick · 2020-02-07T21:21:42Z

At one point @oleg-nenashev had been playing with related stuff.

olivergondza · 2020-02-10T08:32:48Z

Thanks for the reviews so far. I will proceed with integrating this by the end of the week provided there is no negative feedback.

uhafner · 2020-02-18T22:02:45Z

I tried to follow your example for the warnings plugin. I copied the tests to a new module and started them, but the tests do not start up due to an exception. I downloaded the kerberos plugin and here I also get the same exception:

[ERROR] com.sonymobile.jenkins.plugins.kerberossso.KerberosSsoTest.explicitTicketAuth  Time elapsed: 4.938 s  <<< ERROR!
com.google.inject.ProvisionException: 
Unable to provision, see the following errors:

1) Error in custom provider, org.openqa.selenium.remote.UnreachableBrowserException: Could not start a new session. Possible causes are invalid address of the remote server or browser start-up failure.
Build info: version: '3.14.0', revision: 'aacccce0', time: '2018-08-02T20:19:58.91Z'
System info: host: 'Ullis-MBP-3.fritz.box', ip: '2003:cf:f32:d300:402b:fd9d:8528:49d0%en0', os.name: 'Mac OS X', os.arch: 'x86_64', os.version: '10.15.3', java.version: '1.8.0_242'
Driver info: driver.version: RemoteWebDriver
  at org.jenkinsci.test.acceptance.FallbackConfig.createWebDriver(FallbackConfig.java:293) (via modules: org.jenkinsci.test.acceptance.guice.World -> com.google.inject.util.Modules$OverrideModule -> com.google.inject.util.Modules$OverrideModule -> org.jenkinsci.test.acceptance.FallbackConfig)
  at org.jenkinsci.test.acceptance.FallbackConfig.createWebDriver(FallbackConfig.java:293) (via modules: org.jenkinsci.test.acceptance.guice.World -> com.google.inject.util.Modules$OverrideModule -> com.google.inject.util.Modules$OverrideModule -> org.jenkinsci.test.acceptance.FallbackConfig)
  while locating org.openqa.selenium.WebDriver
    for field at org.jenkinsci.test.acceptance.junit.DiagnosticRule.driver(DiagnosticRule.java:19)
  while locating org.jenkinsci.test.acceptance.junit.DiagnosticRule

1 error
Caused by: org.openqa.selenium.remote.UnreachableBrowserException: 
Could not start a new session. Possible causes are invalid address of the remote server or browser start-up failure.
Build info: version: '3.14.0', revision: 'aacccce0', time: '2018-08-02T20:19:58.91Z'
System info: host: 'Ullis-MBP-3.fritz.box', ip: '2003:cf:f32:d300:402b:fd9d:8528:49d0%en0', os.name: 'Mac OS X', os.arch: 'x86_64', os.version: '10.15.3', java.version: '1.8.0_242'
Driver info: driver.version: RemoteWebDriver
Caused by: java.net.ConnectException: Failed to connect to /127.0.0.1:64632
Caused by: java.net.ConnectException: Connection refused (Connection refused)

[INFO] 
[INFO] Results:
[INFO] 
[ERROR] Errors: 
[ERROR]   KerberosSsoTest.basicAuth » Provision Unable to provision, see the following e...
[ERROR]   KerberosSsoTest.explicitBasicAuth » Provision Unable to provision, see the fol...
[ERROR]   KerberosSsoTest.explicitTicketAuth » Provision Unable to provision, see the fo...
[ERROR]   KerberosSsoTest.kerberosTicket » Provision Unable to provision, see the follow...
[ERROR]   KerberosSsoTest.kerberosTicketWithBasicAuthEnabled » Provision Unable to provi...
[ERROR]   KerberosSsoTest.kerberosTicketWithBasicAuthEnabledJcasc » Provision Unable to ...
[INFO] 
[ERROR] Tests run: 6, Failures: 0, Errors: 6, Skipped: 0

Is macOS not supported for these kind of tests? ATH is running fine on macOS though.

olivergondza · 2020-02-19T15:15:24Z

@uhafner, good point, i have not kept macos in mind. I suppose native host networking will never work on macos and win so we probably have to rethink that. (This depend on native host networking as the browser inside container needs to be able to access Jenkins running on the host)

jglick · 2020-02-19T22:06:53Z

the browser inside container needs to be able to access Jenkins running on the host

FWIW I find it more straightforward to just run mvn test, and thus (with the default controller) Jenkins, inside a sibling container as well.

olivergondza · 2020-02-20T08:33:19Z

@jglick, wrapping Jenkins to a container (too) has a problem that it needs to reach to MockUpdateCenter (at least) running inside surefire process (on the host). Moving surefire into a container (the same or different from Jenkins) is something I have abandoned for its hideousness:

So yeah, this is something to explore.

uhafner · 2020-02-20T11:04:45Z

I see. I can run the tests locally on macOS using the standard mechanism (e.g. JUT server), that works even from IntelliJ. During the CI build of my plugin I can switch to your new approach...

olivergondza · 2020-02-20T12:51:30Z

@uhafner, that might be the simplest way to go on Mac. Note just changing the variable from <BROWSER>firefox-container</BROWSER> to <BROWSER>firefox</BROWSER> should do the trick.

jglick · 2020-02-24T15:01:17Z

wrapping Jenkins to a container (too) has a problem that it needs to reach to MockUpdateCenter (at least) running inside surefire process (on the host). Moving surefire into a container (the same or different from Jenkins)

FWIW I was not proposing any of these things. Rather, moving the entire mvn test inside a (maven:3.6.3-jdk-8, e.g.) container.

Explore ATH run against containerized selenium

f749d27

olivergondza mentioned this pull request Jan 28, 2020

WIP: Prototype nested ATH invocation from Jenkins plugin #555

Closed

olivergondza added 2 commits January 28, 2020 13:59

Fix Docker class initialization

739b748

Polish a little bit

e47a009

olivergondza mentioned this pull request Feb 3, 2020

Move ath to module to escape the dependency hell jenkinsci/openstack-cloud-plugin#288

Merged

olivergondza added 3 commits February 4, 2020 15:05

Log the order of cleanup operations

c8b1c2b

Describe the selenium container cleanup operation

468ef60

Document how containerized selenium is supposed to be used by plugins

445f854

olivergondza requested review from Vlatombe, amuniz and vilacides and removed request for Vlatombe February 5, 2020 07:53

amuniz approved these changes Feb 5, 2020

View reviewed changes

amuniz mentioned this pull request Feb 5, 2020

Move ath to module to escape the dependency hell jenkinsci/kerberos-sso-plugin#19

Merged

olivergondza mentioned this pull request Feb 7, 2020

Move ath to module to escape the dependency hell jenkinsci/jms-messaging-plugin#164

Merged

Allocate random port for control and vnc ports

68120f6

vilacides approved these changes Feb 7, 2020

View reviewed changes

jglick requested a review from oleg-nenashev February 7, 2020 21:21

olivergondza merged commit 8a5acca into jenkinsci:master Feb 14, 2020

olivergondza deleted the prototype-containerized-selenium branch February 14, 2020 09:07

uhafner mentioned this pull request Feb 25, 2020

Move UI tests from ATH to plugin jenkinsci/warnings-ng-plugin#396

Closed

uhafner mentioned this pull request Mar 10, 2020

Move UI tests from ATH to plugin jenkinsci/warnings-ng-plugin#419

Merged

uhafner mentioned this pull request Feb 18, 2022

Feature/UI tests jenkinsci/junit-plugin#342

Draft

6 tasks

uhafner mentioned this pull request Mar 21, 2023

Additional test coverage #1053

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Explore ATH run against containerized selenium #556

Explore ATH run against containerized selenium #556

olivergondza commented Jan 28, 2020 •

edited

Loading

olivergondza commented Feb 5, 2020

amuniz left a comment

amuniz Feb 5, 2020

vilacides Feb 7, 2020

jglick commented Feb 7, 2020

olivergondza commented Feb 10, 2020

uhafner commented Feb 18, 2020

olivergondza commented Feb 19, 2020

jglick commented Feb 19, 2020

olivergondza commented Feb 20, 2020

uhafner commented Feb 20, 2020

olivergondza commented Feb 20, 2020

jglick commented Feb 24, 2020

Explore ATH run against containerized selenium #556

Explore ATH run against containerized selenium #556

Conversation

olivergondza commented Jan 28, 2020 • edited Loading

olivergondza commented Feb 5, 2020

amuniz left a comment

Choose a reason for hiding this comment

amuniz Feb 5, 2020

Choose a reason for hiding this comment

vilacides Feb 7, 2020

Choose a reason for hiding this comment

jglick commented Feb 7, 2020

olivergondza commented Feb 10, 2020

uhafner commented Feb 18, 2020

olivergondza commented Feb 19, 2020

jglick commented Feb 19, 2020

olivergondza commented Feb 20, 2020

uhafner commented Feb 20, 2020

olivergondza commented Feb 20, 2020

jglick commented Feb 24, 2020

olivergondza commented Jan 28, 2020 •

edited

Loading