Sauce_Ruby_Integration_Roadmap

Sauce Ruby Integration

Process

• One repo per shippable software unit
• Pull request model using forks and a commit history of logical changes
• On each pull request:
  • Tests must pass before merging
  • Code coverage must not regress
  • Style check must pass
• Readme with badges to easily see latest released version, build status, and coverage percentage

Technology

• Travis CI for continuous integration
• RSpec for tests
• coveralls for code coverage
• rubocop for static code analysis / style check

Scope

• New gem for file parallel execution based on parallel_tests
• New gem for test parallel execution based on parallel_split_test
• New gem for Sauce integration (may be based on sauce_whisk)
  • Pass metadata to Sauce such as pass/fail
  • Integrate properly with Sauce Jenkins plugin
  • Support modern Ruby and RSpec
  • Retry support (may be based on rspec-rerun)

Finished

• New parallel execution gem test-queue-split (based on test-queue)
• sauce_ruby gem replacement sauce_rspec
• rspec-retry works as is for retrying failed tests

Timeline

Estimated timeline is 2-3 months. Gems will be transferred to the Sauce Labs organization after being used in a production test suite to ensure quality.

Future

After the initial scope is complete, it may make sense to migrate the following gems to Sauce as well. These gems already exist and are at 100% code coverage.

• sauce_platforms - Easily specify Sauce Platforms and get an immediate exception upon an invalid request.
• rspec_flake - Measure flakiness of rspec tests on Sauce. Parses rspec_junit output to identify per file times (useful for improving parallel_tests performance)
• rspec_junit - JUnit XML reporting for use with the Sauce Labs Jenkins plugin. Includes timing information about individual tests.