Handle exceptions properly in SpringJUnit4ClassRunner [SPR-12613]

[spring-projects-issues]

Sam Brannen opened SPR-12613 and commented

Status Quo

SpringJUnit4ClassRunner has extended BlockJUnit4ClassRunner for several years and inherited default JUnit behavior from BlockJUnit4ClassRunner. At times, the default behavior needed to be copied and modified for special needs of the Spring TestContext Framework. For example, the BlockJUnit4ClassRunner.runChild() method has always been overridden in order to support Spring's @IfProfileValue annotation for ignored tests.

In JUnit 4.9, the implementation of BlockJUnit4ClassRunner.runChild() changed such that the invocation of BlockJUnit4ClassRunner.methodBlock() is no longer wrapped in a try-catch block. The end result is a change in backwards compatibility. Namely, any exceptions thrown from methodBlock() are no longer properly associated with the currently executing test method. Consequently:

1. such failures are displayed as "unrooted" (i.e., not associated with the corresponding test method)
2. all subsequent test methods for the current test class are not executed at all
3. JUnit test listeners are not notified of such failures or of any subsequent test methods that should have been executed

In conjunction with #16527, SpringJUnit4ClassRunner was updated to use the aforementioned changes to BlockJUnit4ClassRunner.runChild() introduced in JUnit 4.9. As a consequence, this introduces a regression in SpringJUnit4ClassRunner which can be seen in test failures in TimedSpringRunnerTests.

---

Deliverables

1. Ensure that any exceptions thrown within SpringJUnit4ClassRunner.methodBlock() are properly associated with the currently executing test.
2. Inform the JUnit team of the regression they introduced in JUnit 4.9

---

Affects: 4.1 GA

Issue Links:

• Require JUnit 4.9 or higher in the TestContext framework [SPR-11908] #16527 Require JUnit 4.9 or higher in the TestContext framework

Referenced from: commits b81c522, 7d171e6

[spring-projects-issues]

Sam Brannen commented

Fixed as described in GitHub commit b81c522:

Handle exceptions properly in SpringJUnit4ClassRunner

JUnit 4.9 introduced a regression in BlockJUnit4ClassRunner.runChild()
such that exceptions thrown from methodBlock() cause the current test
execution to abort immediately. As a result, the failing test method is
unrooted, and subsequent test methods are never invoked. Furthermore,
RunListeners registered with JUnit are not properly notified.

In conjunction with #16527, SpringJUnit4ClassRunner was updated to
use the aforementioned changes to BlockJUnit4ClassRunner.runChild().
Consequently, SpringJUnit4ClassRunner now suffers from the same
regression.

This commit addresses this issue by ensuring that any exceptions thrown
during the invocation of methodBlock() are properly wrapped in a JUnit
Fail Statement.

[spring-projects-issues]

Sam Brannen commented

Backported to 4.1.5.

[spring-projects-issues]

Sam Brannen commented

Informed JUnit team by raising issue junit-team/junit4#1066.