Skip to content
This repository

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Selenium driver 

nzakas edited this page · 6 revisions

The YUI Test Selenium Driver is a utility to allow automated execution of JavaScript tests written in YUI Test. In order to use the Selenium Driver, you must have access to a Selenium Server (you can download the server software from SeleniumHQ).

The YUI Test Selenium Driver works by contacting a Selenium Remote Control Server and specifying which test page URLs to load in which browsers. When the test page URL is loaded, the JavaScript tests are executed, and once complete, the results are retrieved and then output into files. The files can then be picked up by a build system, such as Hudson, in order to allow reporting.

Installation

The YUI Test Selenium Driver requires Java 5 (1.5) or higher.

To begin, download the Selenium Java Client Driver from https://github.com/yui/yuitest/blob/master/java/lib/selenium-java-client-driver.jar. Place the JAR file into your JRE's /lib/ext/ directory.

The Selenium Driver also requires the latest versions of YUI Test. You can get those from the YUI 2 and YUI 3 GitHub projects.

General Usage

There are several steps to using the YUI Test Selenium Driver:

  1. Write JavaScript tests using YUI Test.
  2. Create a page to run your JavaScript Tests. The tests should be run automatically on page load.
  3. Setup a Selenium server.
  4. Use YUI Test Selenium Driver to run your tests against various browsers via Selenium.

Test Pages

A test page is an HTML page containing the code to be tested as well as YUI Test to execute the tests. In order to ensure that the test page runs correctly, follow these steps:

  1. Include the YUI Test JavaScript file appropriate for your version of YUI.
  2. Add test suites/test cases to the TestRunner object.
  3. The tests must be run automatically by the page (Selenium Driver will not start them for you). Ensure that TestRunner.run() is executed either during window.onload or while the page is loading.
  4. If you're using YUI Test 3, ensure that there's a global Y object and that Y.Test is accessible by the time all tests complete. This should not be an issue for those using YUI Test 2.

Command Line

Usage: java -jar yuitest-selenium-driver-x.y.z.jar [options] [test files]

Global Options
  -h, --help                Displays this information.
  --browsers <browsers>     Run tests in these browsers (comma-delimited).
  --conf <file>             Load options from <file>.
  --coveragedir <dir>       Output coverage files to <dir>.
  --erroronfail             Indicates that a test failure should cause
                            an error to be reported to the console.
  --host <host>             Use the Selenium host <host>.
  --port <port>             Use <port> port on the Selenium host.
  --resultsdir <dir>        Output test result files to <dir>.
  --silent                  Don't output test results to the console.
  --tests <file>            Loads test info from <file>.
  -v, --verbose             Display informational messages and warnings.

The options are:

  • The --browsers option specifies a comma-delimited list of Selenium browsers to test. By default, this is set to *firefox. You can use any number of browsers, such as *firefox,*iexplore,*opera. This can be set in a configuration file using the key selenium.browsers.
  • The --conf option specifies a properties file that can specify most of the command line options.
  • The --coveragedir option specifies the output directory for code coverage results.
  • The --erroronfail option specifies that if a test fails, it should cause an error such that the Selenium Driver exits immediately with a code of 1 instead of continuing on to other tests and exiting with a code of 0. This can be set in a configuration file setting the key console.erroronfail to 1.
  • The --host option specifies the Selenium or Selenium Grid host on which to run the tests. The default value is localhost. This can be set in a configuration file using the key selenium.host.
  • The --port option specifies the port on the Selenium or Selenium Grid host on which to run the tests. The default value is 4444. This can be set in a configuration file using the key selenium.port.
  • The --resultsdir option specifies the output directory for test results.
  • The --silent option specifies that you don't want test results output to the console. By default, test results are output onto the console. This can be set in a configuration file setting the key console.mode to silent.
  • The --tests option specifies a XML file with test information (see below for format). When a tests file is specified, you need not pass in any test URLs on the command line. The tests file can be either a local file or a web address. Omitting this option means that test URLs must be passed on the command line.

In all cases, command line options override properties read in via --conf.

Examples

The following runs the driver for a single URL assuming YUI Test 2:

java -jar yuitest-selenium-driver.jar http://www.example.com/tests/test_ui

The following is the same as above, but makes sure the driver looks for YUI Test 3:

java -jar yuitest-selenium-driver.jar --yuitest 3 http://www.example.com/tests/test_ui

The following runs the driver for two URLs on two browsers assuming YUI Test 2:

java -jar yuitest-selenium-driver.jar --browsers *firefox,*iexplore http://www.example.com/tests/test_ui http://www.example.com/tests/test_util

The following runs the driver against tests specified in an XML file:

java -jar yuitest-selenium-driver.jar --tests tests.xml

Tests XML File

The --tests option can be used to specify an XML file containing test information. This contains information about the tests to run and has the following general format:

<?xml version="1.0"?>
<yuitest>
    <tests base="http://www.example.com/tests/" timeout="10000">
        <url>test_core</url>
        <url timeout="30000">test_util</url>
        <url>test_ui</url>
    </tests>
</yuitest>

There are only three possible elements to use in the file:

  1. <yuitest> - document element.
  2. <tests> - second level element, indicates a group of tests with common settings. The base attribute is used to indicate the base location for these tests and can be safely omitted if the tests don't share the same base. The timeout attribute indicates the maximum time (in milliseconds) that the Selenium Driver should wait for the tests to complete. If the test doesn't complete in that amount of time, an error is thrown. You can also specify a version attribute on this element to override the one provided on <yuitest>. There can be any number of <tests> elements under <yuitest>.
  3. <url> - third level element, indicates the URL to hit to run tests. If there is a base attribute on the parent <tests> element, then that value is prepended. Each <url> element inherits values for version and timeout from its ancestors but also can override them individually if necessary. There can be any number of <url> elements in each <tests> element.

Config File

You can use a config file to store commonly used configuration options in order to avoid putting them on the command line. A config file is a simple Java properties file comprised of key-value pairs. The following keys are supported:

  • selenium.host - the hostname of the Selenium or Selenium Grid server.
  • selenium.port - the port of the Selenium or Selenium Grid server.
  • selenium.browsers - a comma-delimited list of browsers on which to run the tests. The browser names are those used by Selenium and so any valid browser name to Selenium is also valid here.
  • results.format - the format in which to save the test results. These correspond to the test formats provided by YUI Test. There are three possible values: YUITestXML, TAP, and JUnitXML (default).
  • coverage.format - the format in which to save the coverage information. These correspond to the coverage formats provided by YUI Test. Possible values are LCOV (default), GCOV and HTML.
  • results.outputdir - the directory in which to save the test results.
  • coverage.outputdir - the directory in which to save the coverage information.
  • results.filename - the filename format for test results. There are three variables you can specify in the filename format: {browser} inserts the Selenium browser name, {name} inserts the test page name, and {date:<format>} inserts the date in the specified format, where is a SimpleDateFormat
  • console.mode - the console output mode. Possible values are normal and silent. This setting determines whether or not test results are output to the console.
  • console.erroronfail - indicates if the Selenium Driver should exit with a status code of 1 when a test fails.

Troubleshooting

Class not found error

You see this:

Exception in thread "main" java.lang.NoClassDefFoundError: com/thoughtworks/selenium/Selenium
    at com.yahoo.platform.yuitest.selenium.YUITestSeleniumDriver.main(YUITestSeleniumDriver.java:146)

This means that you've not copied the Selenium Java client driver into the appropriate JRE location. Please see Installation for more details.

Failed to start new browser session

[ERROR] Could not start Selenium session: Failed to start new browser session:      java.lang.RuntimeException: Firefox 3 could not be found in the path!
Please add the directory containing ''firefox-bin' or 'firefox'' to your PATH environment
variable, or explicitly specify a path to Firefox 3 like this:
*firefox3/blah/blah/firefox-bin

This sometimes occurs in RHEL environments. The problem is that Selenium (not the Selenium Driver) cannot find Firefox on the system and therefore can't start it up. The error message actually describes the exact steps you should follow: either add the full path of the Firefox executable to your PATH environment variable or specify the full path of the browser using the selenium.browsers configuration parameter in the configuration file:

selenium.browsers = *firefox /usr/bin/firefox

Timeout issue

By default, Selenium will only wait for 30 seconds for an action to complete. In the case of the Selenium Driver, this means a page must complete all tests within 30 seconds. If you need longer, you'll need to set this up when launching the Selenium server:

java -jar selenium-server.jar -timeout 120

More information: http://release.seleniumhq.org/selenium-core/1.0/reference.html#setTimeout

Something went wrong with that request. Please try again.