Integrating Opan Sauce cross browser automated testing

SauceLabs allows automated tests to be run on a variety of browser/OS combinations, which may be used to expose cross-browser incompatibility in the project. Open Sauce is free for open source projects. To get started integrating SauceLabs support in the project, follow these steps:

1. Add SauceLabs environment variables. In config/application.yml, add your SauceLabs username and password. A SauceLabs Open Sauce account can be created for free if your project is open source (create an account). The Access Key can be found in the lower-left of the Sauce Labs web panel after logging in.

Additionally, add a RUN_ON_SAUCE environment variable with the value or true (run on SauceLabs), or false or commented out (don't run on SauceLabs):

SAUCE_USERNAME: <YOUR_SAUCE_LABS_USERNAME>
SAUCE_ACCESS_KEY: <YOUR_SAUCE_LABS_ACCESS_KEY>
# Whether to run tests through SauceLabs.
RUN_ON_SAUCE: 'true'

2. Add the SauceLabs gems. In the Gemfile, add the SauceLabs gems (in the :test group):

group :test do
  …
  # For integrating Open Sauce (SauceLabs) for testing.
  gem 'sauce'
  # Sauce Connect is required by default.
  gem 'sauce-connect'
  # Run SauceLabs tests across multiple concurrent VM instances on SauceLabs.
  gem 'parallel_tests'
  …
end

3. Don't require webmock gem. Also in the Gemfile's :test group add :require => false to the webmock gem, like the following:

Note: update the gem version to whatever version the project is currently using.

group :test do
  …
  gem 'webmock', '~> 1.20.0', :require => false
  …
end

4. Install gems. From the command line, run the bundle command to install the sauce labs gems.

5. Add Sauce test helper. Add file spec/support/sauce_helper.rb with the following contents:

# You should edit this file with the browsers you wish to use.
# For options, check out http://saucelabs.com/docs/platforms.
if ENV['RUN_ON_SAUCE']
  require "sauce"
  require "sauce/capybara"
  Sauce.config do |c|
    c[:name] = 'Automated test'
    c[:start_tunnel] = true
    c[:start_local_application] = false
    c[:application_host] = ENV['DOMAIN_NAME']
    c[:application_port] = 4000
    c[:browsers] = [
        ["OSX 10.6", "Firefox", 17],
        ["Windows 7", "Opera", 10],
        ["Windows 7", "Firefox", 20]
      ]
  end
end

4. Edit Capybara support file. Update Capybara configuration file (spec/support/capybara.rb) to conditionally use SauceLabs driver if RUN_ON_SAUCE environment variable is present:

BEFORE

Capybara.configure do |config|
  config.javascript_driver = :poltergeist
  config.default_wait_time = 30
  config.always_include_port = true
end

AFTER

Capybara.configure do |config|
  if ENV['RUN_ON_SAUCE']
    config.default_driver = :sauce
    config.javascript_driver = :sauce
  else
    config.javascript_driver = :poltergeist
    config.default_wait_time = 30
    config.always_include_port = true
  end
end

5. Edit VCR support file. Update VCR configuration file (spec/support/vcr.rb) to only use VCR when not using SauceLabs:

BEFORE

require 'webmock/rspec'
require 'vcr'

VCR.configure do |config|
  …
end

AFTER

# Disable mocking of URL requests if using Sauce Labs for testing.
unless ENV["RUN_ON_SAUCE"]
  require 'webmock/rspec'
  require 'vcr'

  VCR.configure do |config|
    …
  end
end

6. Flag tests to run on SauceLabs. Add :sauce => ENV['RUN_ON_SAUCE'] to all tests you want to run on SauceLabs. Tests that are not flagged to run on Sauce may fail when SauceLabs testing is run. For example:

feature 'homepage search', :sauce => ENV['RUN_ON_SAUCE'] do
  …
end

7. Restart environment. Stop and start the web server (from the command line, press ctrl+c and enter rails s -p 4000, respectively). Stop and start spring (stop spring and start spring, respectively).

8. Run tests. From the command line, run the tests with script/test and observe the output from SauceLabs (console will display [Connecting to Sauce Labs...]). You can also log into your SauceLabs account to see the tests running.