Assumptions don't work in tests with expected exceptions #121

Closed
petur opened this Issue Jul 13, 2010 · 4 comments

Comments

Projects
None yet
3 participants

petur commented Jul 13, 2010

The following test should be ignored, but fails with JUnit 4.8.2:

@Test(expected = UnsupportedOperationException.class)
public void testExpectedVsAssumption() {
    Assume.assumeTrue(false);
}

This test succeeds if run with @runwith(JUnit4ClassRunner.class). The reason it works with the older runner, but fails with the new runner seems to be that org.junit.internal.runners.MethodRoadie.runTestMethod() has special handling for AssumptionViolatedException, but this handling is missing from org.junit.internal.runners.statements.ExpectException.evaluate().

petur commented Jul 13, 2010

It would also be nice if ExpectException didn't wrap AssertionError. I would expect the following test to fail with an AssertionError, but it fails with an Exception instead:

@Test(expected = UnsupportedOperationException.class)
public void testExpectedVsAssertion() {
    Assert.assertTrue(false);
}

The stack trace is:

java.lang.Exception: Unexpected exception, expected<java.lang.UnsupportedOperationException> but was<java.lang.AssertionError>
at org.junit.internal.runners.statements.ExpectException.evaluate(ExpectException.java:28)
at org.junit.runners.BlockJUnit4ClassRunner.runNotIgnored(BlockJUnit4ClassRunner.java:79)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:71)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:49)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184)
at org.junit.runners.ParentRunner.run(ParentRunner.java:236)
Caused by: java.lang.AssertionError:
at org.junit.Assert.fail(Assert.java:91)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertTrue(Assert.java:54)
at AssumptionVsExpectedTest.testExpectedVsAssertion(AssumptionVsExpectedTest.java:17)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
at org.junit.internal.runners.statements.ExpectException.evaluate(ExpectException.java:21)
... 15 more

petur commented Jul 13, 2010

The same problem appears when using rules instead. This fails, but should probably succeed:

@Rule
public ExpectedException ex = ExpectedException.none();

@Test
public void testExpectedVsAssumption() {
    ex.expect(UnsupportedOperationException.class);

    Assume.assumeTrue(false);
}

In this case, it is easy to move the assumption above the expect, but that is not possible when using @test(expected = xxx.class).

Contributor

stefanbirkner commented Sep 10, 2011

The introductory issue is a duplicate of issue #98 and is already fixed in 4.9.

@stefanbirkner stefanbirkner added a commit to stefanbirkner/junit that referenced this issue Sep 11, 2011

@stefanbirkner stefanbirkner Don't handle AssumptionViolatedException.
An AssumptionViolatedException thrown by the Statement is rethrown.
Therefore tests are ignored if assume... fails, regardless whether
an expected exception is thrown or not.

Fix #121. Thanks to petur for discovering this bug.
6263b97

@stefanbirkner stefanbirkner added a commit to stefanbirkner/junit that referenced this issue Sep 11, 2011

@stefanbirkner stefanbirkner Don't handle AssertionError.
An AssertionError thrown by the Statement is rethrown. Therefore tests
fail because of the AssertionError and not because of a wrong expected
exception.

Fix #121. Thanks to petur for discovering this bug.
6e6bac0

@stefanbirkner stefanbirkner added a commit to stefanbirkner/junit that referenced this issue Sep 21, 2011

@stefanbirkner stefanbirkner Fixes gh-121 (ExpectedException handles JUnit exceptions)
ExpectedException rule no longer handles AssertionErrors and
AssumptionViolatedExceptions by default. This means that the
rule doesn't intercept if your test fails because of an violated
assertion or assumption.

If you want to test assertions or assumptions you have to tell
the rule to handle such exceptions via handleAssertionErrors()
or handleAssumptionViolatedExceptions().
128553f
Contributor

stefanbirkner commented Sep 21, 2011

I fixed ExpectedException to work with assume and don't wrap AssertionErrors. AssertionErrors are still wrapped if you're using the expected=Foo.class notation. Please use ExpectedException to avoid wrapping.

dsaff closed this in c4279e4 Apr 9, 2012

@stefanbirkner stefanbirkner added a commit to stefanbirkner/junit that referenced this issue Aug 17, 2013

@stefanbirkner stefanbirkner Handle expected AssertionError and AssumptionViolatedException.
Fixes #687. #121 is still fixed. (Pull request #323 was too rigid.)
b6c581a

@stefanbirkner stefanbirkner added a commit to stefanbirkner/junit that referenced this issue Aug 17, 2013

@stefanbirkner stefanbirkner Handle expected AssertionError and AssumptionViolatedException.
Fixes #687. #121 is still fixed. (Pull request #323 was too rigid.)
fc7bcdb

@stefanbirkner stefanbirkner added a commit to stefanbirkner/junit that referenced this issue Aug 17, 2013

@stefanbirkner stefanbirkner Handle expected AssertionError and AssumptionViolatedException.
Fixes #687. #121 is still fixed. (Pull request #323 was too rigid.)
9fef81b

@stefanbirkner stefanbirkner added a commit to stefanbirkner/junit that referenced this issue Aug 17, 2013

@stefanbirkner stefanbirkner Handle expected AssertionError and AssumptionViolatedException.
Fixes #687. #121 is still fixed. (Pull request #323 was too rigid.)
33553ab

@stefanbirkner stefanbirkner added a commit to stefanbirkner/junit that referenced this issue Sep 3, 2013

@stefanbirkner stefanbirkner Revert changes of 128553f. Fixes #687.
Still fixes #121, but the original fix was superfluous. Added documentation
about handling assumes and asserts in conjunction with the
ExpectedException rule.
3716b59

@stefanbirkner stefanbirkner added a commit to stefanbirkner/junit that referenced this issue Sep 3, 2013

@stefanbirkner stefanbirkner Revert changes of 128553f. Fixes #687.
Still fixes #121, but the original fix was superfluous. Added documentation
about handling assumes and asserts in conjunction with the
ExpectedException rule.
fea1fd6

@stefanbirkner stefanbirkner added a commit to stefanbirkner/junit that referenced this issue Oct 16, 2013

@stefanbirkner stefanbirkner Revert changes of 128553f. Fixes #687.
Still fixes #121, but the original fix was superfluous. Added documentation
about handling assumes and asserts in conjunction with the
ExpectedException rule.
cdbca3c

@stefanbirkner stefanbirkner added a commit to stefanbirkner/junit that referenced this issue Oct 16, 2013

@stefanbirkner stefanbirkner Revert changes of 128553f. Fixes #687.
Still fixes #121, but the original fix was superfluous. Added documentation
about handling assumes and asserts in conjunction with the
ExpectedException rule.
9da2901

@stefanbirkner stefanbirkner added a commit to stefanbirkner/junit that referenced this issue Oct 16, 2013

@stefanbirkner stefanbirkner Revert changes of 128553f. Fixes #687.
Still fixes #121, but the original fix was superfluous. Added documentation
about handling assumes and asserts in conjunction with the
ExpectedException rule.
42b8dcb

@stefanbirkner stefanbirkner added a commit to stefanbirkner/junit that referenced this issue Oct 16, 2013

@stefanbirkner stefanbirkner Revert changes of 128553f. Fixes #687.
Still fixes #121, but the original fix was superfluous. Added documentation
about handling assumes and asserts in conjunction with the
ExpectedException rule.
bb13b31

@stefanbirkner stefanbirkner added a commit to stefanbirkner/junit that referenced this issue Jan 23, 2014

@stefanbirkner stefanbirkner Revert changes of 128553f. Fixes #687.
Still fixes #121, but the original fix was superfluous. Added documentation
about handling assumes and asserts in conjunction with the
ExpectedException rule.
fe4d61c

@josephw josephw added a commit to josephw/hamcrest-junit that referenced this issue Jun 1, 2015

@josephw josephw Bring across ExpectedException from JUnit 4.12.
This class's behaviour changed in JUnit 4.12
(mainly due to junit-team/junit4#121).

Bring across the fix, and its tests, along with class
reformatting.
1cb6e34

@josephw josephw added a commit to josephw/hamcrest-junit that referenced this issue Jun 1, 2015

@josephw josephw Bring across ExpectedException from JUnit 4.12.
This class's behaviour changed in JUnit 4.12
(mainly due to junit-team/junit4#121).

Bring across the fix, and its tests, along with class
reformatting.
af84ed8
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment