Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make ScalaWebTest compatible with all implementations of Selenium WebDriver #74

Closed
DaniRey opened this issue Mar 6, 2019 · 7 comments
Closed
Milestone

Comments

@DaniRey
Copy link
Collaborator

@DaniRey DaniRey commented Mar 6, 2019

If we do not depend on HtmlUnits HTML parsing, we can use other Drivers as well and even extract the gauge logic completely. Ideally, we can compile this for Scala.js and use it in other tools like TestCafé. This will not be possible, as we have to parse HTML (see next comment). Most probably a rewrite for JS is the better option, if we want to use Gauges in TestCafé.

@DaniRey DaniRey changed the title Make parsing of HTML part of ScalaWebTest instead of using HtmlUnit Make ScalaWebTest compatible with all implementations of Selenium WebDriver Mar 27, 2019
@DaniRey
Copy link
Collaborator Author

@DaniRey DaniRey commented Mar 27, 2019

This is a challenging task because:

  • org.openqa.selenium.WebElement is very limited. There is no way to query its parent, siblings or children (no DOM traversal)
  • WebDriver does not provide access to Response Headers and Response Code

Because of this, a few intermediate steps are needed.

  • not being able to use com.gargoylesoftware.htmlunit.html.HtmlElement means we have to parse the HTML in ScalaWebTest
  • not many options exist to parse HTML. So far JSoup seems to be the best option
  • to get a hold of Response Headers and Response Code we have to send all requests through a proxy, which is under our control. We can then ask the proxy to get this information. (inspiration https://github.com/wkeeling/selenium-wire)

DaniRey added a commit that referenced this issue Mar 27, 2019
…Driver #74

 - this is work in progress
 - it's currently coupled to chrome
 - implementations for response code/headers where removed
@DaniRey
Copy link
Collaborator Author

@DaniRey DaniRey commented Apr 2, 2019

I think the final solution should provide three different kinds of Browsers

  • HtmlUnitExtended (default)
  • Selenium
  • ProxiedSelenium

All three should implement a common interface, but similar to JDBC drivers, some methods might not be implemented, pointing to alternatives which implement said method.

HtmlUnitExtended: This webdriver is headless. It uses HtmlUnit and doesn't hide that fact.

No system prerequisites Response Codes Response Headers Screenshots JavaScript
Yes Yes Yes No Limited (RhinoJs)

Selenium: This might be Chrome, Firefox,... controlled via SeleniumWebDriver API

No system prerequisites Response Codes Response Headers Screenshots JavaScript
Browser and WebDriver installed No No Yes Yes

Selenium Proxied: Again Chrome, Firefox... controlled via SeleniumWebDriver API and using a proxy, which is under our control. This allows ScalaWebTest to read the response code and response headers. This information is not exposed by the Selenium WebDriver API and never will (see SeleniumHQ/selenium-google-code-issue-archive#141 (comment)).

No system prerequisites Response Codes Response Headers Screenshots JavaScript
Browser and WebDriver installed and Proxy running Yes Yes Yes Yes

DaniRey added a commit that referenced this issue Apr 2, 2019
DaniRey added a commit that referenced this issue Apr 2, 2019
@DaniRey
Copy link
Collaborator Author

@DaniRey DaniRey commented Apr 2, 2019

Following the reasoning from ashley.leyba in SeleniumHQ/selenium-google-code-issue-archive#141 (comment)

If we attempt to make WebDriver the silver bullet of web testing, then it will suffer in overall quality.

  1. driver.get(url) blocks until the browser has loaded the page. In the case of a login redirect, what status/headers would you want returned? By the semantics of the API, we would return the response for the final loaded page. You would end up with a 200 instead of the 302 you're looking for.
  1. If you want to test HTTP request/response headers, then write an appropriately sized test and test those directly.

We should probably not add Selenium Proxied to ScalaWebTest. I can currently not think of any reasonable use case, where one would require to use Chrome/Firefox/... to test a response code or response header. For those tests, HtmlUnitExtended should be enough. Therefore I would concentrate our efforts on HtmlUnitExtended and Selenium. The ProxiedSelenium variant might follow as a module, but I doubt we will ever need it.

DaniRey added a commit that referenced this issue Apr 2, 2019
@DaniRey
Copy link
Collaborator Author

@DaniRey DaniRey commented Apr 8, 2019

We should try to use or integrate with https://github.com/bonigarcia/webdrivermanager

DaniRey added a commit that referenced this issue May 5, 2019
…Driver #74

 - this is work in progress
 - fetch value for webdriver.chrome.driver from environment and system properties
 - using configMap from ScalaTest was discarded, because configMap is only initialized when using org.scalatest.tools.Runner to run the test
 - additionally using configMap would require additional indirection, because we have to delay the initialization of the webdriver until we receive the configMap
 - therefore we decided to use System.getProperty and System.getenv and provide the Configurable abstraction to access the information
DaniRey added a commit that referenced this issue May 27, 2019
…Driver #74

 - this is work in progress
 - change BaseConfiguration is now abstract and requires a browser specific implementation
 - Selenium browsers report configurations, which they cannot apply as error, and configurations which are superfluous as warnings
 - added slf4-simple as dependency for scalawebtest-integration
 - switched back to HtmlUnit as default browser for the integration tests
 - reactivated the ResponseAccessors, they do now require WebClientExposingDriver
 - AemTweaks no longer overrides config, but instead provides additional config via implicit conversion
DaniRey added a commit that referenced this issue May 28, 2019
…Driver #74

 - correctly handle JSON responses in Chrome (stripping the HTML wrapper)
 - added basic tests, to verify functionality across all browsers
 - moved comments from HtmlUnitConfiguration to BaseConfiguration
 - with this commit the change should be good enough to be merged back to develop
DaniRey added a commit that referenced this issue May 28, 2019
…Driver #74

 - removed explicit use of HtmlUnitWebDriver in some tests (this was introduced as a temporary solution for this PR)
 - changed initialization of Logger to always use getClass.getName (benefit: we know which test produced this log entry, downside: we do no longer see, which trait contained the code that produced this log entry)
 - with this commit the change should be good enough to be merged back to develop
DaniRey added a commit that referenced this issue May 28, 2019
…Driver #74

 - ChromeDriverServiceRunner can either forward to an existing service runner, or start it's own
DaniRey added a commit that referenced this issue May 28, 2019
…Driver #74

 - environment variables in Linux may not contain 'dot', therefore changing webdriver.chrome.driver.service.url to WEBDRIVER_CHROME_DRIVER_SERVICE_URL
DaniRey added a commit that referenced this issue May 28, 2019
…Driver #74

 - change WEBDRIVER_CHROME_DRIVER_SERVICE_URL to a valid URL
DaniRey added a commit that referenced this issue May 28, 2019
…Driver #74

 - try RemoteWebDriver with empty ChromeOptions
DaniRey added a commit that referenced this issue May 28, 2019
…Driver #74

 - Configurable tries property names in upper-case, with dots and hyphens replaced by underscore, when searching an according environment variable, because environment variables under linux may not contain dot or hyphen and are typically in uppercase
 - Try to start chrome driver service on travis from Scala instead of using the one started in the before_script
DaniRey added a commit that referenced this issue May 28, 2019
…Driver #74

 - cleanup travis.yml after finding a working combination of configurations and code
DaniRey added a commit that referenced this issue May 29, 2019
…Driver #74

 - switch to BeforeAndAfterAllConfigMap
 - Configurable now also reads parameters from the ConfigMap
 - ChromeDriverService initialization is now delayed until the first test needs the according driver
 - SeleniumChrome arguments are now configurable
DaniRey added a commit that referenced this issue May 29, 2019
DaniRey added a commit that referenced this issue May 29, 2019
…Driver #74

 - rename and re-enable WebDriverConfigFixtures as WebClientExposingDriverConfigFixtures
 - add type annotation in WebClientExposingHtmlUnit and remove the implicit keyword for the webdriver (only the one in IntegrationSpec should be implicit)
@DaniRey DaniRey added this to the 3.0.0 milestone May 29, 2019
DaniRey added a commit that referenced this issue May 29, 2019
…Driver #74

 - cleanup commented code
 - remove superfluous empty parenthesis
 - implement prettyPrint
@DaniRey
Copy link
Collaborator Author

@DaniRey DaniRey commented May 31, 2019

ScalaWebTest is now compatible with any Selenium WebDriver. It still uses HtmlUnit as it's default. Also some features, like verifying the responseCode or disabling JavaScript, are only available with HtmlUnit (because the Selenium WebDriver API does not allow to control or access this information).

At the moment we provide an implementation to use Chrome. Additional implementations are planned. At least Firefox, probably Safari as well. Our plan is to support all browsers, which we can test with Travis. We do not want to add implementations, which we cannot test as part of our CI build. As ScalaWebTest now works against the Selenium WebDriver API, users are free to plug any Selenium WebDriver, when using ScalaWebTest.

DaniRey added a commit that referenced this issue Jun 1, 2019
- merge host and loginPath/basePath into config.useBaseUri/loginConfig.useLoginUri, this makes configuration consistent (always in config object) and allows for cleaner configuration via command line arguments
- config.baseUri can be overwritten with the system.property(scalawebtest.base.uri), the environment variable (scalawebtest.base.uri or SCALAWEBTEST_BASE_URI), or the run argument (scalawebtest.base.uri)
- loginConfig.loginUri can be overwritten with the system.property(scalawebtest.login.uri), the environment variable (scalawebtest.login.uri or SCALAWEBTEST_LOGIN_URI), or the run argument (scalawebtest.login.uri)
- the extension points (or methods) login, beforeLogin and afterLogin now receive the configMap as well. This allows to read run arguments and make use of the Configurable trait in those methods
- reactivate the BasicAuthLogin trait, which got broken during the implementation for #74
DaniRey added a commit that referenced this issue Jun 9, 2019
…Driver #74

 - add support for Firefox via Mozilla Gecko Driver
DaniRey added a commit that referenced this issue Jun 9, 2019
…Driver #74

 - add support for Firefox via Mozilla Gecko Driver
DaniRey added a commit that referenced this issue Jun 9, 2019
…Driver #74

 - add support for Firefox via Mozilla Gecko Driver
DaniRey added a commit that referenced this issue Jun 9, 2019
…Driver #74

 - add support for Firefox via Mozilla Gecko Driver
DaniRey added a commit that referenced this issue Jun 9, 2019
…Driver #74

 - add support for Firefox via Mozilla Gecko Driver
DaniRey added a commit that referenced this issue Jun 10, 2019
…Driver #74

 - add support for Firefox via Mozilla Gecko Driver
DaniRey added a commit that referenced this issue Jun 10, 2019
…Driver #74

 - add support for Firefox via Mozilla Gecko Driver
 - "--no-remote" should disable dbus communication, therefore not start the dbus-daemon and prevent warnings from the dbus-deamon because no DISPLAY is defined
@DaniRey
Copy link
Collaborator Author

@DaniRey DaniRey commented Jun 10, 2019

Firefox implementation was tried with https://github.com/unic/ScalaWebTest/blob/feature/add_firefox

The implementation itself works well, but the build on TravisCI is very unstable with Firefox added.
The log is cluttered with

(firefox:6123): GConf-WARNING **: Client failed to connect to the D-BUS daemon:
Unable to autolaunch a dbus-daemon without a $DISPLAY for X11

And Firefox sometimes fails with

[info] org.scalawebtest.integration.browser.SeleniumFirefoxSpec *** ABORTED ***
[info]   org.openqa.selenium.JavascriptException: TypeError: document.documentElement is null
[info] Build info: version: '3.141.59', revision: 'e82be7d358', time: '2018-11-14T08:17:03'
[info] System info: host: 'travis-job-ce3fd6ff-0693-4bcc-bc88-8aaf7d82f424', ip: '127.0.1.1', os.name: 'Linux', os.arch: 'amd64', os.version: '4.15.0-1028-gcp', java.version: '1.8.0_191'
[info] Driver info: org.scalawebtest.core.browser.SeleniumFirefox$CleanedPageSourceFirefoxDriver
[info] Capabilities {acceptInsecureCerts: true, browserName: firefox, browserVersion: 67.0.1, javascriptEnabled: true, moz:accessibilityChecks: false, moz:buildID: 20190529130856, moz:geckodriverVersion: 0.24.0, moz:headless: true, moz:processID: 5203, moz:profile: /tmp/rust_mozprofile.vikkzN..., moz:shutdownTimeout: 60000, moz:useNonSpecCompliantPointerOrigin: false, moz:webdriverClick: true, pageLoadStrategy: normal, platform: LINUX, platformName: LINUX, platformVersion: 4.15.0-1028-gcp, rotatable: false, setWindowRect: true, strictFileInteractability: false, timeouts: {implicit: 0, pageLoad: 300000, script: 30000}, unhandledPromptBehavior: dismiss and notify}
[info] Session ID: adae5d6e-965e-4d67-9311-71ec8b431a22
[info]   at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
[info]   at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
[info]   at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
[info]   at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
[info]   at org.openqa.selenium.remote.http.W3CHttpResponseCodec.createException(W3CHttpResponseCodec.java:187)
[info]   at org.openqa.selenium.remote.http.W3CHttpResponseCodec.decode(W3CHttpResponseCodec.java:122)
[info]   at org.openqa.selenium.remote.http.W3CHttpResponseCodec.decode(W3CHttpResponseCodec.java:49)
[info]   at org.openqa.selenium.remote.HttpCommandExecutor.execute(HttpCommandExecutor.java:158)
[info]   at org.openqa.selenium.remote.service.DriverCommandExecutor.execute(DriverCommandExecutor.java:83)
[info]   at org.openqa.selenium.remote.RemoteWebDriver.execute(RemoteWebDriver.java:552)

@DaniRey
Copy link
Collaborator Author

@DaniRey DaniRey commented Jun 11, 2019

As Firefox isn't stable yet on Travis CI, we will not add it to 3.0.0.

For now, everyone interested may just grab the implementation from https://github.com/unic/ScalaWebTest/tree/feature/add_firefox

As far as we can tell now, the problem is not in our implementation, but rather in our configuration of the build agent on Travis.

Adding Firefox is from now on tracked in #85

@DaniRey DaniRey closed this Jun 11, 2019
DaniRey added a commit that referenced this issue Jul 9, 2020
…Driver #74

 - add support for Firefox via Mozilla Gecko Driver
DaniRey added a commit that referenced this issue Jul 9, 2020
…Driver #74

 - add support for Firefox via Mozilla Gecko Driver
 - "--no-remote" should disable dbus communication, therefore not start the dbus-daemon and prevent warnings from the dbus-deamon because no DISPLAY is defined
DaniRey added a commit that referenced this issue Jul 9, 2020
…Driver #74

- add support for Firefox via Mozilla Gecko Driver
DaniRey added a commit that referenced this issue Jul 9, 2020
…Driver #74

- add support for Firefox via Mozilla Gecko Driver
DaniRey added a commit that referenced this issue Jul 9, 2020
…Driver #74

- add support for Firefox via Mozilla Gecko Driver
DaniRey added a commit that referenced this issue Jul 9, 2020
…Driver #74

- add support for Firefox via Mozilla Gecko Driver
DaniRey added a commit that referenced this issue Jul 9, 2020
…Driver #74

- add support for Firefox via Mozilla Gecko Driver
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
1 participant