Retry Failed Or Skipped Testcases

As we know, Web UI autotest are not as stable as other kinds of tests such as: interface test, unit test.

So we modified TestNg source code to retry failed/skipped testcases, and built a new TestNg jar file, see details here.

How to use

Step 1, modify demo.xml as follow:

Here reruntimes="2" means a testcase has 2 chances of retry. A failed/skipped testcase will retry until success or exhaust its chances, and result of the last retry will be taken account into TestNg's report. In this way most failed/skipped testcases which actually are caused by instability can be filtered out.

Good to know:

The testcases retry in the unit of <test>, such as: TestNg and TestNgClone.

Step 2, use Dagger's TestNg jar file to run testcases (In Eclipse testcases are run with TestNg plugin's jar file). Here we use Apache Ant and prepare a simple build.xml under Dagger's root path, just have a try :-)

<?xml version="1.0"?>
<project default="demo">

	<taskdef resource="testngtasks" classpath="lib/testng-re-html.jar" />

	<path id="cpath">
		<fileset dir="lib">
			<include name="*.jar" />
			<include name="**/*.jar"/>
		</fileset>
	</path>

	<path id="runpath">
		<pathelement location="bin" />
		<path refid="cpath" />
	</path>

	<target name="compile">
		<delete dir="bin" />
		<mkdir dir="bin" />
		<javac destdir="bin" classpathref="cpath" srcdir="src" debug="true" encoding="UTF8">
		</javac>
		<copy file="src/log4j.properties" tofile="bin/log4j.properties" />
	</target>

	<target name="demo" depends="compile">
		<delete dir="test-output" />
		<mkdir dir="test-output" />
		<testng outputDir="test-output" classpathref="runpath" >
			<xmlfileset dir="test-xml" includes="demo.xml" />
		</testng>
	</target>

</project>