Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make ScalaWebTest compatible with all implementations of Selenium WebDriver #74

Closed
DaniRey opened this issue Mar 6, 2019 · 7 comments

Comments

Projects
None yet
1 participant
@DaniRey
Copy link
Member

commented Mar 6, 2019

If we do not depend on HtmlUnits HTML parsing, we can use other Drivers as well and even extract the gauge logic completely. Ideally, we can compile this for Scala.js and use it in other tools like TestCafé. This will not be possible, as we have to parse HTML (see next comment). Most probably a rewrite for JS is the better option, if we want to use Gauges in TestCafé.

@DaniRey DaniRey changed the title Make parsing of HTML part of ScalaWebTest instead of using HtmlUnit Make ScalaWebTest compatible with all implementations of Selenium WebDriver Mar 27, 2019

@DaniRey

This comment has been minimized.

Copy link
Member Author

commented Mar 27, 2019

This is a challenging task because:

  • org.openqa.selenium.WebElement is very limited. There is no way to query its parent, siblings or children (no DOM traversal)
  • WebDriver does not provide access to Response Headers and Response Code

Because of this, a few intermediate steps are needed.

  • not being able to use com.gargoylesoftware.htmlunit.html.HtmlElement means we have to parse the HTML in ScalaWebTest
  • not many options exist to parse HTML. So far JSoup seems to be the best option
  • to get a hold of Response Headers and Response Code we have to send all requests through a proxy, which is under our control. We can then ask the proxy to get this information. (inspiration https://github.com/wkeeling/selenium-wire)

DaniRey added a commit that referenced this issue Mar 27, 2019

Make ScalaWebTest compatible with all implementations of Selenium Web…
…Driver #74

 - this is work in progress
 - it's currently coupled to chrome
 - implementations for response code/headers where removed
@DaniRey

This comment has been minimized.

Copy link
Member Author

commented Apr 2, 2019

I think the final solution should provide three different kinds of Browsers

  • HtmlUnitExtended (default)
  • Selenium
  • ProxiedSelenium

All three should implement a common interface, but similar to JDBC drivers, some methods might not be implemented, pointing to alternatives which implement said method.

HtmlUnitExtended: This webdriver is headless. It uses HtmlUnit and doesn't hide that fact.

No system prerequisites Response Codes Response Headers Screenshots JavaScript
Yes Yes Yes No Limited (RhinoJs)

Selenium: This might be Chrome, Firefox,... controlled via SeleniumWebDriver API

No system prerequisites Response Codes Response Headers Screenshots JavaScript
Browser and WebDriver installed No No Yes Yes

Selenium Proxied: Again Chrome, Firefox... controlled via SeleniumWebDriver API and using a proxy, which is under our control. This allows ScalaWebTest to read the response code and response headers. This information is not exposed by the Selenium WebDriver API and never will (see SeleniumHQ/selenium-google-code-issue-archive#141 (comment)).

No system prerequisites Response Codes Response Headers Screenshots JavaScript
Browser and WebDriver installed and Proxy running Yes Yes Yes Yes

DaniRey added a commit that referenced this issue Apr 2, 2019

DaniRey added a commit that referenced this issue Apr 2, 2019

@DaniRey

This comment has been minimized.

Copy link
Member Author

commented Apr 2, 2019

Following the reasoning from ashley.leyba in SeleniumHQ/selenium-google-code-issue-archive#141 (comment)

If we attempt to make WebDriver the silver bullet of web testing, then it will suffer in overall quality.

  1. driver.get(url) blocks until the browser has loaded the page. In the case of a login redirect, what status/headers would you want returned? By the semantics of the API, we would return the response for the final loaded page. You would end up with a 200 instead of the 302 you're looking for.
  1. If you want to test HTTP request/response headers, then write an appropriately sized test and test those directly.

We should probably not add Selenium Proxied to ScalaWebTest. I can currently not think of any reasonable use case, where one would require to use Chrome/Firefox/... to test a response code or response header. For those tests, HtmlUnitExtended should be enough. Therefore I would concentrate our efforts on HtmlUnitExtended and Selenium. The ProxiedSelenium variant might follow as a module, but I doubt we will ever need it.

DaniRey added a commit that referenced this issue Apr 2, 2019

@DaniRey

This comment has been minimized.

Copy link
Member Author

commented Apr 8, 2019

We should try to use or integrate with https://github.com/bonigarcia/webdrivermanager

DaniRey added a commit that referenced this issue May 5, 2019

Make ScalaWebTest compatible with all implementations of Selenium Web…
…Driver #74

 - this is work in progress
 - fetch value for webdriver.chrome.driver from environment and system properties
 - using configMap from ScalaTest was discarded, because configMap is only initialized when using org.scalatest.tools.Runner to run the test
 - additionally using configMap would require additional indirection, because we have to delay the initialization of the webdriver until we receive the configMap
 - therefore we decided to use System.getProperty and System.getenv and provide the Configurable abstraction to access the information

DaniRey added a commit that referenced this issue May 27, 2019

Make ScalaWebTest compatible with all implementations of Selenium Web…
…Driver #74

 - this is work in progress
 - change BaseConfiguration is now abstract and requires a browser specific implementation
 - Selenium browsers report configurations, which they cannot apply as error, and configurations which are superfluous as warnings
 - added slf4-simple as dependency for scalawebtest-integration
 - switched back to HtmlUnit as default browser for the integration tests
 - reactivated the ResponseAccessors, they do now require WebClientExposingDriver
 - AemTweaks no longer overrides config, but instead provides additional config via implicit conversion

DaniRey added a commit that referenced this issue May 28, 2019

Make ScalaWebTest compatible with all implementations of Selenium Web…
…Driver #74

 - correctly handle JSON responses in Chrome (stripping the HTML wrapper)
 - added basic tests, to verify functionality across all browsers
 - moved comments from HtmlUnitConfiguration to BaseConfiguration
 - with this commit the change should be good enough to be merged back to develop

DaniRey added a commit that referenced this issue May 28, 2019

Make ScalaWebTest compatible with all implementations of Selenium Web…
…Driver #74

 - removed explicit use of HtmlUnitWebDriver in some tests (this was introduced as a temporary solution for this PR)
 - changed initialization of Logger to always use getClass.getName (benefit: we know which test produced this log entry, downside: we do no longer see, which trait contained the code that produced this log entry)
 - with this commit the change should be good enough to be merged back to develop

DaniRey added a commit that referenced this issue May 28, 2019

Make ScalaWebTest compatible with all implementations of Selenium Web…
…Driver #74

 - ChromeDriverServiceRunner can either forward to an existing service runner, or start it's own

DaniRey added a commit that referenced this issue May 28, 2019

Make ScalaWebTest compatible with all implementations of Selenium Web…
…Driver #74

 - environment variables in Linux may not contain 'dot', therefore changing webdriver.chrome.driver.service.url to WEBDRIVER_CHROME_DRIVER_SERVICE_URL

DaniRey added a commit that referenced this issue May 28, 2019

Make ScalaWebTest compatible with all implementations of Selenium Web…
…Driver #74

 - change WEBDRIVER_CHROME_DRIVER_SERVICE_URL to a valid URL

DaniRey added a commit that referenced this issue May 28, 2019

Make ScalaWebTest compatible with all implementations of Selenium Web…
…Driver #74

 - try RemoteWebDriver with empty ChromeOptions

DaniRey added a commit that referenced this issue May 28, 2019

Make ScalaWebTest compatible with all implementations of Selenium Web…
…Driver #74

 - Configurable tries property names in upper-case, with dots and hyphens replaced by underscore, when searching an according environment variable, because environment variables under linux may not contain dot or hyphen and are typically in uppercase
 - Try to start chrome driver service on travis from Scala instead of using the one started in the before_script

DaniRey added a commit that referenced this issue May 28, 2019

Make ScalaWebTest compatible with all implementations of Selenium Web…
…Driver #74

 - cleanup travis.yml after finding a working combination of configurations and code

DaniRey added a commit that referenced this issue May 29, 2019

Make ScalaWebTest compatible with all implementations of Selenium Web…
…Driver #74

 - switch to BeforeAndAfterAllConfigMap
 - Configurable now also reads parameters from the ConfigMap
 - ChromeDriverService initialization is now delayed until the first test needs the according driver
 - SeleniumChrome arguments are now configurable

DaniRey added a commit that referenced this issue May 29, 2019

DaniRey added a commit that referenced this issue May 29, 2019

Make ScalaWebTest compatible with all implementations of Selenium Web…
…Driver #74

 - rename and re-enable WebDriverConfigFixtures as WebClientExposingDriverConfigFixtures
 - add type annotation in WebClientExposingHtmlUnit and remove the implicit keyword for the webdriver (only the one in IntegrationSpec should be implicit)

@DaniRey DaniRey added this to the 3.0.0 milestone May 29, 2019

DaniRey added a commit that referenced this issue May 29, 2019

Make ScalaWebTest compatible with all implementations of Selenium Web…
…Driver #74

 - cleanup commented code
 - remove superfluous empty parenthesis
 - implement prettyPrint
@DaniRey

This comment has been minimized.

Copy link
Member Author

commented May 31, 2019

ScalaWebTest is now compatible with any Selenium WebDriver. It still uses HtmlUnit as it's default. Also some features, like verifying the responseCode or disabling JavaScript, are only available with HtmlUnit (because the Selenium WebDriver API does not allow to control or access this information).

At the moment we provide an implementation to use Chrome. Additional implementations are planned. At least Firefox, probably Safari as well. Our plan is to support all browsers, which we can test with Travis. We do not want to add implementations, which we cannot test as part of our CI build. As ScalaWebTest now works against the Selenium WebDriver API, users are free to plug any Selenium WebDriver, when using ScalaWebTest.

@DaniRey DaniRey referenced this issue May 31, 2019

Closed

Update Documentation for 3.0.0 #79

9 of 9 tasks complete

DaniRey added a commit that referenced this issue Jun 1, 2019

Changing host or port configs by passing them via runner arguments #12
- merge host and loginPath/basePath into config.useBaseUri/loginConfig.useLoginUri, this makes configuration consistent (always in config object) and allows for cleaner configuration via command line arguments
- config.baseUri can be overwritten with the system.property(scalawebtest.base.uri), the environment variable (scalawebtest.base.uri or SCALAWEBTEST_BASE_URI), or the run argument (scalawebtest.base.uri)
- loginConfig.loginUri can be overwritten with the system.property(scalawebtest.login.uri), the environment variable (scalawebtest.login.uri or SCALAWEBTEST_LOGIN_URI), or the run argument (scalawebtest.login.uri)
- the extension points (or methods) login, beforeLogin and afterLogin now receive the configMap as well. This allows to read run arguments and make use of the Configurable trait in those methods
- reactivate the BasicAuthLogin trait, which got broken during the implementation for #74

@DaniRey DaniRey added the enhancement label Jun 7, 2019

DaniRey added a commit that referenced this issue Jun 9, 2019

Make ScalaWebTest compatible with all implementations of Selenium Web…
…Driver #74

 - add support for Firefox via Mozilla Gecko Driver

DaniRey added a commit that referenced this issue Jun 9, 2019

Make ScalaWebTest compatible with all implementations of Selenium Web…
…Driver #74

 - add support for Firefox via Mozilla Gecko Driver

DaniRey added a commit that referenced this issue Jun 9, 2019

Make ScalaWebTest compatible with all implementations of Selenium Web…
…Driver #74

 - add support for Firefox via Mozilla Gecko Driver

DaniRey added a commit that referenced this issue Jun 9, 2019

Make ScalaWebTest compatible with all implementations of Selenium Web…
…Driver #74

 - add support for Firefox via Mozilla Gecko Driver

DaniRey added a commit that referenced this issue Jun 9, 2019

Make ScalaWebTest compatible with all implementations of Selenium Web…
…Driver #74

 - add support for Firefox via Mozilla Gecko Driver

DaniRey added a commit that referenced this issue Jun 10, 2019

Make ScalaWebTest compatible with all implementations of Selenium Web…
…Driver #74

 - add support for Firefox via Mozilla Gecko Driver

DaniRey added a commit that referenced this issue Jun 10, 2019

Make ScalaWebTest compatible with all implementations of Selenium Web…
…Driver #74

 - add support for Firefox via Mozilla Gecko Driver
 - "--no-remote" should disable dbus communication, therefore not start the dbus-daemon and prevent warnings from the dbus-deamon because no DISPLAY is defined
@DaniRey

This comment has been minimized.

Copy link
Member Author

commented Jun 10, 2019

Firefox implementation was tried with https://github.com/unic/ScalaWebTest/blob/feature/add_firefox

The implementation itself works well, but the build on TravisCI is very unstable with Firefox added.
The log is cluttered with

(firefox:6123): GConf-WARNING **: Client failed to connect to the D-BUS daemon:
Unable to autolaunch a dbus-daemon without a $DISPLAY for X11

And Firefox sometimes fails with

[info] org.scalawebtest.integration.browser.SeleniumFirefoxSpec *** ABORTED ***
[info]   org.openqa.selenium.JavascriptException: TypeError: document.documentElement is null
[info] Build info: version: '3.141.59', revision: 'e82be7d358', time: '2018-11-14T08:17:03'
[info] System info: host: 'travis-job-ce3fd6ff-0693-4bcc-bc88-8aaf7d82f424', ip: '127.0.1.1', os.name: 'Linux', os.arch: 'amd64', os.version: '4.15.0-1028-gcp', java.version: '1.8.0_191'
[info] Driver info: org.scalawebtest.core.browser.SeleniumFirefox$CleanedPageSourceFirefoxDriver
[info] Capabilities {acceptInsecureCerts: true, browserName: firefox, browserVersion: 67.0.1, javascriptEnabled: true, moz:accessibilityChecks: false, moz:buildID: 20190529130856, moz:geckodriverVersion: 0.24.0, moz:headless: true, moz:processID: 5203, moz:profile: /tmp/rust_mozprofile.vikkzN..., moz:shutdownTimeout: 60000, moz:useNonSpecCompliantPointerOrigin: false, moz:webdriverClick: true, pageLoadStrategy: normal, platform: LINUX, platformName: LINUX, platformVersion: 4.15.0-1028-gcp, rotatable: false, setWindowRect: true, strictFileInteractability: false, timeouts: {implicit: 0, pageLoad: 300000, script: 30000}, unhandledPromptBehavior: dismiss and notify}
[info] Session ID: adae5d6e-965e-4d67-9311-71ec8b431a22
[info]   at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
[info]   at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
[info]   at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
[info]   at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
[info]   at org.openqa.selenium.remote.http.W3CHttpResponseCodec.createException(W3CHttpResponseCodec.java:187)
[info]   at org.openqa.selenium.remote.http.W3CHttpResponseCodec.decode(W3CHttpResponseCodec.java:122)
[info]   at org.openqa.selenium.remote.http.W3CHttpResponseCodec.decode(W3CHttpResponseCodec.java:49)
[info]   at org.openqa.selenium.remote.HttpCommandExecutor.execute(HttpCommandExecutor.java:158)
[info]   at org.openqa.selenium.remote.service.DriverCommandExecutor.execute(DriverCommandExecutor.java:83)
[info]   at org.openqa.selenium.remote.RemoteWebDriver.execute(RemoteWebDriver.java:552)
@DaniRey

This comment has been minimized.

Copy link
Member Author

commented Jun 11, 2019

As Firefox isn't stable yet on Travis CI, we will not add it to 3.0.0.

For now, everyone interested may just grab the implementation from https://github.com/unic/ScalaWebTest/tree/feature/add_firefox

As far as we can tell now, the problem is not in our implementation, but rather in our configuration of the build agent on Travis.

Adding Firefox is from now on tracked in #85

@DaniRey DaniRey closed this Jun 11, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.