Cucumber rerun formatter example
You have a whole bunch of Cucumber features using some flavour of Selenium-Webdriver to test your webapp, you have been very conscientious and:
- Implemented sane retry logic to find UI elements
- Isolated your UI using stub backends to guard against unexpected data
- Added spoonfuls of helpful debug logging to highlight issues
But your tests still appear to now and again be nondeterministic and people are getting frustrated.
This can happen, UI testing is hard and no matter how much you defend against random failure peculiarities of the runtime environment can conspire against you.
In this instance it would be handy to have 'one more go' when you get some failures and Cucumber's rerun formatter allows this.
This is about the simplest example I could come up with.
The test will fail 50% of the time.
You can run the test using
bundle exec rake
It will run the test for a second time if it fails and should terminate with an appropriate exit code.
You can see some examples of this working in the projects Travis build.
These aren't my ideas: