exception in method and dependsonmethods tests still running: results all messed up

[DaveBrad]

The same test class is run using different versions of eclipse plugs 2011-06-02 vs 2011-06-24. The class throws an exception in the first test case with the other test cases using dependonmethods to force a sequence run of the TCs.

TC1, TC2 depending on TC1, TC3 depending on TC2, TC4 depending on TC3,......

Exception in TC1, TC2 is skipped but TC3, TC4 are run.....

0602 works as expected: eclipse37/plugins/og.testng.eclipse_6.0.1.20110602_1209/lib/testng.jar

          GUI display--                   results: passed: 0 failed 1: skipped 5

         Console display--
         ===============================================
         Default test
         Tests run: 6, Failures: 1, Skips: 5
         ===============================================

0624 gives bad results: eclipse37/plugins/og.testng.eclipse_6.0.1.20110624_1020/lib/testng.jar

               GUI display--                    results:  passed: 4  failed 1: skipped 1

              Console display--
              ===============================================
              Default test
              Tests run: 6, Failures: 1, Skips: 1
               ===============================================

It appears the exception failure in the first test case does not abort the test class from continuing (as test2 depend-on test1, test3 depends-on test2,.......)

code and output below

import org.testng.annotations.Test;

@test
public class ExceptionThrower {
@test
public void initTesting(){
System.out.println("expect this");
throw new RuntimeException();
}
/////////////////
@test (dependsOnMethods="initTesting")
public void dummyTest1(){
System.out.println("should fail to get here dummyTest1");
}
/////////////////
@test (dependsOnMethods="dummyTest1")
public void dummyTest2(){
System.out.println("should fail to get here dummyTest2");
}
/////////////////
@test (dependsOnMethods="dummyTest2")
public void dummyTest3(){
System.out.println("should fail to get here dummyTest3");
}
/////////////////
@test (dependsOnMethods="dummyTest3")
public void dummyTest4(){
System.out.println("should fail to get here dummyTest4");
}
/////////////////
@test (dependsOnMethods="dummyTest4")
public void dummyTest5(){
System.out.println("should fail to get here dummyTest5");
}
}

eclipse37/plugins/og.testng.eclipse_6.0.1.20110602_1209/lib/testng.jar

RemoteTestNG starting
expect this
FAILED: initTesting
java.lang.RuntimeException
at ExceptionThrower.initTesting(ExceptionThrower.java:10)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:81)
at org.testng.internal.Invoker.invokeMethod(Invoker.java:673)
at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:842)
at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1166)
at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:125)
at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:109)
at org.testng.TestRunner.runWorkers(TestRunner.java:1150)
at org.testng.TestRunner.privateRun(TestRunner.java:752)
at org.testng.TestRunner.run(TestRunner.java:603)
at org.testng.SuiteRunner.runTest(SuiteRunner.java:329)
at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:324)
at org.testng.SuiteRunner.privateRun(SuiteRunner.java:286)
at org.testng.SuiteRunner.run(SuiteRunner.java:235)
at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52)
at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:86)
at org.testng.TestNG.runSuitesSequentially(TestNG.java:1049)
at org.testng.TestNG.runSuitesLocally(TestNG.java:974)
at org.testng.TestNG.run(TestNG.java:905)
at org.testng.remote.RemoteTestNG.run(RemoteTestNG.java:111)
at org.testng.remote.RemoteTestNG.initAndRun(RemoteTestNG.java:203)
at org.testng.remote.RemoteTestNG.main(RemoteTestNG.java:174)

SKIPPED: dummyTest1
SKIPPED: dummyTest2
SKIPPED: dummyTest3
SKIPPED: dummyTest4
SKIPPED: dummyTest5

Default test

Tests run: 6, Failures: 1, Skips: 5

[TestNG] Time taken by org.testng.reporters.JUnitReportReporter@fe64b9: 16 ms
[TestNG] Time taken by org.testng.reporters.SuiteHTMLReporter@1d8957f: 31 ms
[TestNG] Time taken by org.testng.reporters.XMLReporter@1f9dc36: 0 ms
[TestNG] Time taken by org.testng.reporters.EmailableReporter@1833955: 16 ms
[TestNG] Time taken by [TestListenerAdapter] Passed:0 Failed:0 Skipped:0]: 15 ms
RemoteTestNG finishing: 265 ms

0624 gives bad results: eclipse37/plugins/og.testng.eclipse_6.0.1.20110624_1020/lib/testng.jar

RemoteTestNG starting
expect this
should fail to get here dummyTest2 <---------------------------------------------------------------
should fail to get here dummyTest3
should fail to get here dummyTest4
should fail to get here dummyTest5
PASSED: dummyTest2
PASSED: dummyTest3
PASSED: dummyTest4
PASSED: dummyTest5
FAILED: initTesting
java.lang.RuntimeException
at ExceptionThrower.initTesting(ExceptionThrower.java:10)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:81)
at org.testng.internal.Invoker.invokeMethod(Invoker.java:673)
at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:842)
at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1166)
at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:125)
at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:109)
at org.testng.TestRunner.runWorkers(TestRunner.java:1172)
at org.testng.TestRunner.privateRun(TestRunner.java:757)
at org.testng.TestRunner.run(TestRunner.java:608)
at org.testng.SuiteRunner.runTest(SuiteRunner.java:334)
at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:329)
at org.testng.SuiteRunner.privateRun(SuiteRunner.java:291)
at org.testng.SuiteRunner.run(SuiteRunner.java:240)
at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52)
at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:86)
at org.testng.TestNG.runSuitesSequentially(TestNG.java:1158)
at org.testng.TestNG.runSuitesLocally(TestNG.java:1083)
at org.testng.TestNG.run(TestNG.java:999)
at org.testng.remote.RemoteTestNG.run(RemoteTestNG.java:111)
at org.testng.remote.RemoteTestNG.initAndRun(RemoteTestNG.java:203)
at org.testng.remote.RemoteTestNG.main(RemoteTestNG.java:174)

SKIPPED: dummyTest1

Default test

Tests run: 6, Failures: 1, Skips: 1

[TestNG] Time taken by org.testng.reporters.EmailableReporter@fa7e74: 8 ms
[TestNG] Time taken by org.testng.reporters.JUnitReportReporter@133796: 8 ms
[TestNG] Time taken by org.testng.reporters.SuiteHTMLReporter@7a84e4: 45 ms
[TestNG] Time taken by org.testng.reporters.XMLReporter@1837697: 10 ms
[TestNG] Time taken by [TestListenerAdapter] Passed:0 Failed:0 Skipped:0]: 10 ms
RemoteTestNG finishing: 264 ms

[cbeust]

Fixed in 6.1.1.