Simple example of how to rerun failed tests with Cucumber
Ruby Gherkin
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
config
features
.gitignore
.travis.yml
Gemfile
Gemfile.lock
README.md
Rakefile

README.md

Build Status

Cucumber rerun formatter example

Why?

You have a whole bunch of Cucumber features using some flavour of Selenium-Webdriver to test your webapp, you have been very conscientious and:

  • Implemented sane retry logic to find UI elements
  • Isolated your UI using stub backends to guard against unexpected data
  • Added spoonfuls of helpful debug logging to highlight issues

But your tests still appear to now and again be nondeterministic and people are getting frustrated.

This can happen, UI testing is hard and no matter how much you defend against random failure peculiarities of the runtime environment can conspire against you.

In this instance it would be handy to have 'one more go' when you get some failures and Cucumber's rerun formatter allows this.

The Example

This is about the simplest example I could come up with.

The test will fail 50% of the time.

You can run the test using bundle exec rake

It will run the test for a second time if it fails and should terminate with an appropriate exit code.

You can see some examples of this working in the projects Travis build.

Credit

These aren't my ideas:

http://blog.mattheworiordan.com/post/42587188567/automatically-retrying-failing-non-deterministic http://blog.crowdint.com/2011/08/22/auto-retry-failed-cucumber-tests.html