Be more flexible with selecting a way to execute tests

[amolenaar]

I've been running some tests with Sauce Labs today. It strikes me as annoying that I have to change my start-browser command every time I want to test on another environment.

The dependency on browser can be dealt with by defining it as a variable. However I can test to:

1. local, using web driver
   | start browser | firefox | on url | http://example.saucelabs.com |
2. local, using selenium server (requires server to be started using SeleniumServerFixture
   | start browser | firefox | on url | http://example.saucelabs.com | using remote server |
3. Remotely, using web driver (e.g. to SauceLabs) (Se2 style)
   | Need to come up with something, key-value pairs have to be defined in a Capabilities object and passed to a RemoteWebDriver instance |
4. Remotely, using Selenium server (Se1 style)
   | start browser | { "username": "your name", "access-key": "some-uuid", "os": "Windows 2003", "browser": "firefox", "browser-version": "7", "name": "Testing Selenium 1 from Xebium" } | on url | http://example.saucelabs.com | using remote server on host | ondemand.saucelabs.com | on port | 80 |

For each version different statements are included. This makes it very complex to switch from one config to another (in a CI setting it's not possible even).

One solution would be to use the syntax as shown in option 4. and allow that to be used for option 1 as well (they're different code paths now!). The org.json.json package is already included through Selenium-server, so we already have that dependency. Defining a special words remote-server with a format server:port' will make the| using remote server |part obsolete. Makingremote-serveran empty string would default tolocalhost:4444`.

TODO: How to distinguish between WebDriver and Http (Selenium 1) driver?
How can this be easily configured in a CI setting? Do we want a property file/system property to set this?

[cwortel]

A simple solution that might work is the following

Create several scenario's for each approach one, for instance:
| scenario | start saucelabs |

| start browser | firefox | on url | http://example.saucelabs.com |

| scenario | start local |

| start browser | firefox | on url | http://example.saucelabs.com | using remote server |

etc...

Now in each test (or in your SuiteSetUp) you start with calling this scenario, but rather than using the name, use a variable:
| script |
| ${startScenario} |
| rest of my test |
| stop browser |

When running the test from your CI system, just start with setting the correct variable for the environment, the rest is done automatically.

Regards,

Cirilo

On 3 nov 2011( w 44), at 11:34, Arjan Molenaar wrote:

I've been running some tests with Sauce Labs today. It strikes me as annoying that I have to change my start-browser command every time I want to test on another environment.

The dependency on browser can be dealt with by defining it as a variable. However I can test to:

1. local, using web driver
   | start browser | firefox | on url | http://example.saucelabs.com |
2. local, using selenium server (requires server to be started using SeleniumServerFixture
   | start browser | firefox | on url | http://example.saucelabs.com | using remote server |
3. Remotely, using web driver (e.g. to SauceLabs) (Se2 style)
   | Need to come up with something, key-value pairs have to be defined in a Capabilities object and passed to a RemoteWebDriver instance |
4. Remotely, using Selenium server (Se1 style)
   | start browser | { "username": "your name", "access-key": "some-uuid", "os": "Windows 2003", "browser": "firefox", "browser-version": "7", "name": "Testing Selenium 1 from Xebium" } | on url | http://example.saucelabs.com | using remote server on host | ondemand.saucelabs.com | on port | 80 |

For each version different statements are included. This makes it very complex to switch from one config to another (in a CI setting it's not possible even).

One solution would be to use the syntax as shown in option 4. and allow that to be used for option 1 as well (they're different code paths now!). The org.json.json package is already included through Selenium-server, so we already have that dependency. Defining a special words remote-server with a format server:port' will make the| using remote server |part obsolete. Makingremote-serveran empty string would default tolocalhost:4444`.

TODO: How to distinguish between WebDriver and Http (Selenium 1) driver?

Reply to this email directly or view it on GitHub:
#23

[amolenaar]

Fair enough. But the selenium 2 remote (web driver) functionality exposes yet another interface. Of course we can add a new method to the code, but still, it's not ideal.

Having a strict separation between selenium 1 (remote server) and Selenium 2 is a pre, though.

[amolenaar]

... and how to deal with username and access key? You don't want those directly in your pages.

[amolenaar]

Maven/ant can take care of the user credentials. Apart from that I think Cirilo's approach is the simplest. It makes it explicit in FitNesse what can be executed.