Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Loading…

Fixes gh-121 (ExpectedException handles JUnit exceptions) #323

Merged
merged 5 commits into from

3 participants

@stefanbirkner
Collaborator

ExpectedException rule no longer handles AssertionErrors and
AssumptionViolatedExceptions by default. This means that the
rule doesn't intercept if your test fails because of an violated
assertion or assumption.

If you want to test assertions or assumptions you have to tell
the rule to handle such exceptions via handleAssertionErrors()
or handleAssumptionViolatedExceptions().

@stefanbirkner stefanbirkner Fixes gh-121 (ExpectedException handles JUnit exceptions)
ExpectedException rule no longer handles AssertionErrors and
AssumptionViolatedExceptions by default. This means that the
rule doesn't intercept if your test fails because of an violated
assertion or assumption.

If you want to test assertions or assumptions you have to tell
the rule to handle such exceptions via handleAssertionErrors()
or handleAssumptionViolatedExceptions().
128553f
...ava/org/junit/runner/notification/EventCollector.java
@@ -0,0 +1,58 @@
+package org.junit.runner.notification;
@dsaff Owner
dsaff added a note

It looks like this class is added in two different places?

@stefanbirkner Collaborator

Ooops, I remove this one.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@dsaff dsaff commented on the diff
...t/tests/experimental/rules/ExpectedExceptionTest.java
((83 lines not shown))
+ public ExpectedExceptionTest(Class<?> classUnderTest,
+ Matcher<EventCollector> matcher) {
+ this.classUnderTest= classUnderTest;
+ this.matcher= matcher;
+ }
+
+ @Test
+ public void runTestAndVerifyResult() {
+ EventCollector collector= new EventCollector();
+ JUnitCore core= new JUnitCore();
+ core.addListener(collector);
+ core.run(classUnderTest);
+ assertThat(collector, matcher);
+ }
+
+ public static class EmptyTestExpectingNoException {
@dsaff Owner
dsaff added a note

For years now, I've wanted to be able to write tests like this, but with a hitch: the matcher for results would be attached to the test class, like this:

public static class EmptyTestExpectingNoException {
public static Matcher<?> EXPECTED = everyTestRunSuccessful()

@Rule public ExpectedException thrown= none();
@Test public void throwsNothing() {}
}

Would you be willing and able to rewrite this test class in that style?

@stefanbirkner Collaborator

I give it a try.

@Tibor17
Tibor17 added a note

@David, Stefan

I think that we can the best if we use the Matcher.
I have reported IsThrowable matcher
https://github.com/KentBeck/junit/issues/322
which might be used.

I prefer a generic form rather than a concrete method names handleAssertionErrors(), handleAssumptionViolatedExceptions().

What about to use the rules like this


@Rule
public ExpectedException thrown= new ExpectedException(Matcher);

We can use anyOf() with these matcher. The only thing is to implement IBlock with run() method which throws anything.

@dsaff Owner
dsaff added a note

Tibor,

ExpectedException already has expect(Matcher). What are you proposing to use instead?

@stefanbirkner Collaborator

Finally I created the JUnitSelfTestRunner. See #361.

@dsaff Owner
dsaff added a note

Stefan,

Would you prefer we pull this change in unchanged, or should we finish #361, and use that for these tests here?

@stefanbirkner Collaborator

I would prefer to pull this change in unchanged.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@Tibor17

Hi David,
In a test, where all test case are supposed to throw same, you have to use
@Rule ee = EE.none().expect(Matcher)
because the expect() method is on instance. That's not much verbose form of expression.

Then the most simple form would be just
@Rule ee = new EE(Matcher)

Any other consequent call to expect() makes the macher more narrow as before, which is ok.

It would be a formal change for the variable fMatcher been not null, with a small effort, IMHO.

@dsaff
Owner

Tibor17, I can see your feature request--it seems orthogonal to this pull request, however. Do you not think so?

@Tibor17

Well yes, that one feature req. really can not be applicable. What you think of the proposals regarding @Rule ee = new EE(Matcher) ?

@dsaff
Owner

Tibor17,

I think that's a pretty good idea, but can be handled in another thread.

@dsaff
Owner

Stefan,

Looking back at this, it's troubling that this test would fail:

@Test public void throwExpectedAssertionError() {
thrown.expect(AssertionError.class);
throw new AssertionError();
}

I think I would change the goals of this pull a bit:

1) Don't handle AssertionErrors differently. The error message is a bit weird, but maybe we can handle that differently in a different pull.

2) Handle AssumptionViolatedExceptions as you've implemented here, with one change: have handleAssumptionViolatedExceptions return this, so people can say:

@Rule public void ExpectedException expect = ExpectedException.none().handleAssumptionViolatedExceptions();

Thoughts?

stefanbirkner added some commits
@stefanbirkner stefanbirkner deleted duplicate EventCollector bfd54d6
@stefanbirkner stefanbirkner the two handle... methods are returning the rule
The two methods handleAssertionError() and
handleAssumptionViolatedExceptions() are returning the rule.

Now you can write
  @Rule
  public ExpectedException thrown = none().handleAssertionError();
or
  @Rule
  public ExpectedException thrown = none().handleAssumptionViolatedExceptions()

Thank you David, for this suggestion.
df70c10
@dsaff dsaff commented on the diff
src/main/java/org/junit/rules/ExpectedException.java
@@ -120,17 +168,32 @@ public void evaluate() throws Throwable {
}
}
+ private void optionallyHandleException(Throwable e, boolean handleException)
+ throws Throwable {
+ if (handleException)
+ handleException(e);
+ else
+ throw e;
+
@dsaff Owner
dsaff added a note

Please remove blank line.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
...rg/junit/tests/experimental/rules/EventCollector.java
((2 lines not shown))
+
+import static org.hamcrest.core.IsEqual.equalTo;
+import static org.junit.matchers.JUnitMatchers.both;
+
+import java.util.ArrayList;
+import java.util.List;
+
+import org.hamcrest.BaseMatcher;
+import org.hamcrest.Matcher;
+import org.junit.runner.Description;
+import org.junit.runner.Result;
+import org.junit.runner.notification.Failure;
+import org.junit.runner.notification.RunListener;
+
+class EventCollector extends RunListener {
+ private final List<Description> testRunsStarted= new ArrayList<Description>();
@dsaff Owner
dsaff added a note

Please use f prefix. Sorry, it's the JUnit standard. :-/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
...rg/junit/tests/experimental/rules/EventCollector.java
((72 lines not shown))
+ sb.append(" test runs finished, ");
+ sb.append(testsStarted.size());
+ sb.append(" tests started, ");
+ sb.append(testsFinished.size());
+ sb.append(" tests finished, ");
+ sb.append(failures.size());
+ sb.append(" failures, ");
+ sb.append(assumptionFailures.size());
+ sb.append(" assumption failures, ");
+ sb.append(testIgnored.size());
+ sb.append(" tests ignored");
+
+ return sb.toString();
+ }
+
+ static Matcher<EventCollector> everyTestRunSuccessful() {
@dsaff Owner
dsaff added a note

Please move static Matcher factories to the top of the class.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
...rg/junit/tests/experimental/rules/EventCollector.java
((79 lines not shown))
+ sb.append(assumptionFailures.size());
+ sb.append(" assumption failures, ");
+ sb.append(testIgnored.size());
+ sb.append(" tests ignored");
+
+ return sb.toString();
+ }
+
+ static Matcher<EventCollector> everyTestRunSuccessful() {
+ return both(hasNoFailure()).and(hasNoAssumptionFailure());
+ }
+
+ private static Matcher<EventCollector> hasNumberOfFailures(
+ final int numberOfFailures) {
+ return new BaseMatcher<EventCollector>() {
+ public boolean matches(Object item) {
@dsaff Owner
dsaff added a note

TypeSafeMatcher would simplify this a bit.

@dsaff Owner
dsaff added a note

Did you decide not to use TypeSafeMatcher here?

@stefanbirkner Collaborator

I was not accurate enough. Fixed it today.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@dsaff
Owner

Any progress on this pull? Thanks.

@stefanbirkner
Collaborator

Applied your code review.

@dsaff dsaff merged commit c4279e4 into junit-team:master
@dsaff
Owner

Many thanks!

@stefanbirkner stefanbirkner referenced this pull request from a commit in stefanbirkner/junit
@stefanbirkner stefanbirkner Handle expected AssertionError and AssumptionViolatedException.
Fixes #687. #121 is still fixed. (Pull request #323 was too rigid.)
b6c581a
@stefanbirkner stefanbirkner referenced this pull request from a commit in stefanbirkner/junit
@stefanbirkner stefanbirkner Handle expected AssertionError and AssumptionViolatedException.
Fixes #687. #121 is still fixed. (Pull request #323 was too rigid.)
fc7bcdb
@stefanbirkner stefanbirkner referenced this pull request from a commit in stefanbirkner/junit
@stefanbirkner stefanbirkner Handle expected AssertionError and AssumptionViolatedException.
Fixes #687. #121 is still fixed. (Pull request #323 was too rigid.)
9fef81b
@stefanbirkner stefanbirkner referenced this pull request from a commit in stefanbirkner/junit
@stefanbirkner stefanbirkner Handle expected AssertionError and AssumptionViolatedException.
Fixes #687. #121 is still fixed. (Pull request #323 was too rigid.)
33553ab
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Commits on Sep 22, 2011
  1. @stefanbirkner

    Fixes gh-121 (ExpectedException handles JUnit exceptions)

    stefanbirkner authored
    ExpectedException rule no longer handles AssertionErrors and
    AssumptionViolatedExceptions by default. This means that the
    rule doesn't intercept if your test fails because of an violated
    assertion or assumption.
    
    If you want to test assertions or assumptions you have to tell
    the rule to handle such exceptions via handleAssertionErrors()
    or handleAssumptionViolatedExceptions().
Commits on Sep 27, 2011
  1. @stefanbirkner
  2. @stefanbirkner

    the two handle... methods are returning the rule

    stefanbirkner authored
    The two methods handleAssertionError() and
    handleAssumptionViolatedExceptions() are returning the rule.
    
    Now you can write
      @Rule
      public ExpectedException thrown = none().handleAssertionError();
    or
      @Rule
      public ExpectedException thrown = none().handleAssumptionViolatedExceptions()
    
    Thank you David, for this suggestion.
Commits on Mar 7, 2012
  1. @stefanbirkner

    Apply code review.

    stefanbirkner authored
Commits on Apr 4, 2012
  1. @stefanbirkner
This page is out of date. Refresh to see the latest.
View
102 src/main/java/org/junit/rules/ExpectedException.java
@@ -1,17 +1,18 @@
package org.junit.rules;
import static org.hamcrest.CoreMatchers.instanceOf;
+import static org.junit.Assert.assertThat;
import static org.junit.matchers.JUnitMatchers.both;
import static org.junit.matchers.JUnitMatchers.containsString;
import org.hamcrest.Description;
import org.hamcrest.Matcher;
import org.hamcrest.StringDescription;
-import org.junit.Assert;
+import org.junit.internal.AssumptionViolatedException;
import org.junit.internal.matchers.TypeSafeMatcher;
import org.junit.runners.model.Statement;
/**
- * The ExpectedException Rule allows in-test specification of expected exception
+ * The ExpectedException rule allows in-test specification of expected exception
* types and messages:
*
* <pre>
@@ -22,7 +23,7 @@
*
* &#064;Test
* public void throwsNothing() {
- * // no exception expected, none thrown: passes.
+ * // no exception expected, none thrown: passes.
* }
*
* &#064;Test
@@ -40,11 +41,39 @@
* }
* }
* </pre>
+ *
+ * By default ExpectedException rule doesn't handle AssertionErrors and
+ * AssumptionViolatedExceptions, because such exceptions are used by JUnit. If
+ * you want to handle such exceptions you have to call @link
+ * {@link #handleAssertionErrors()} or @link
+ * {@link #handleAssumptionViolatedExceptions()}.
+ *
+ * <pre>
+ * // These tests all pass.
+ * public static class HasExpectedException {
+ * &#064;Rule
+ * public ExpectedException thrown= ExpectedException.none();
+ *
+ * &#064;Test
+ * public void throwExpectedAssertionError() {
+ * thrown.handleAssertionErrors();
+ * thrown.expect(AssertionError.class);
+ * throw new AssertionError();
+ * }
+ *
+ * &#064;Test
+ * public void throwExpectAssumptionViolatedException() {
+ * thrown.handleAssumptionViolatedExceptions();
+ * thrown.expect(AssumptionViolatedException.class);
+ * throw new AssumptionViolatedException(&quot;&quot;);
+ * }
+ * }
+ * </pre>
*/
public class ExpectedException implements TestRule {
/**
- * @return a Rule that expects no exception to be thrown
- * (identical to behavior without this Rule)
+ * @return a Rule that expects no exception to be thrown (identical to
+ * behavior without this Rule)
*/
public static ExpectedException none() {
return new ExpectedException();
@@ -52,19 +81,34 @@ public static ExpectedException none() {
private Matcher<Object> fMatcher= null;
+ private boolean handleAssumptionViolatedExceptions= false;
+
+ private boolean handleAssertionErrors= false;
+
private ExpectedException() {
-
}
-
+
+ public ExpectedException handleAssertionErrors() {
+ handleAssertionErrors= true;
+ return this;
+ }
+
+ public ExpectedException handleAssumptionViolatedExceptions() {
+ handleAssumptionViolatedExceptions= true;
+ return this;
+ }
+
public Statement apply(Statement base,
org.junit.runner.Description description) {
return new ExpectedExceptionStatement(base);
}
/**
- * Adds {@code matcher} to the list of requirements for any thrown exception.
+ * Adds {@code matcher} to the list of requirements for any thrown
+ * exception.
*/
- // Should be able to remove this suppression in some brave new hamcrest world.
+ // Should be able to remove this suppression in some brave new hamcrest
+ // world.
@SuppressWarnings("unchecked")
public void expect(Matcher<?> matcher) {
if (fMatcher == null)
@@ -74,24 +118,24 @@ public void expect(Matcher<?> matcher) {
}
/**
- * Adds to the list of requirements for any thrown exception that it
- * should be an instance of {@code type}
+ * Adds to the list of requirements for any thrown exception that it should
+ * be an instance of {@code type}
*/
public void expect(Class<? extends Throwable> type) {
expect(instanceOf(type));
}
/**
- * Adds to the list of requirements for any thrown exception that it
- * should <em>contain</em> string {@code substring}
+ * Adds to the list of requirements for any thrown exception that it should
+ * <em>contain</em> string {@code substring}
*/
public void expectMessage(String substring) {
expectMessage(containsString(substring));
}
/**
- * Adds {@code matcher} to the list of requirements for the message
- * returned from any thrown exception.
+ * Adds {@code matcher} to the list of requirements for the message returned
+ * from any thrown exception.
*/
public void expectMessage(Matcher<String> matcher) {
expect(hasMessage(matcher));
@@ -108,10 +152,14 @@ public ExpectedExceptionStatement(Statement base) {
public void evaluate() throws Throwable {
try {
fNext.evaluate();
+ } catch (AssumptionViolatedException e) {
+ optionallyHandleException(e, handleAssumptionViolatedExceptions);
+ return;
+ } catch (AssertionError e) {
+ optionallyHandleException(e, handleAssertionErrors);
+ return;
} catch (Throwable e) {
- if (fMatcher == null)
- throw e;
- Assert.assertThat(e, fMatcher);
+ handleException(e);
return;
}
if (fMatcher != null)
@@ -120,17 +168,31 @@ public void evaluate() throws Throwable {
}
}
+ private void optionallyHandleException(Throwable e, boolean handleException)
+ throws Throwable {
+ if (handleException)
+ handleException(e);
+ else
+ throw e;
+ }
@dsaff Owner
dsaff added a note

Please remove blank line.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
+
+ private void handleException(Throwable e) throws Throwable {
+ if (fMatcher == null)
+ throw e;
+ assertThat(e, fMatcher);
+ }
+
private Matcher<Throwable> hasMessage(final Matcher<String> matcher) {
return new TypeSafeMatcher<Throwable>() {
public void describeTo(Description description) {
description.appendText("exception with message ");
description.appendDescriptionOf(matcher);
}
-
+
@Override
public boolean matchesSafely(Throwable item) {
return matcher.matches(item.getMessage());
}
};
}
-}
+}
View
4 src/test/java/org/junit/tests/AllTests.java
@@ -25,7 +25,7 @@
import org.junit.tests.experimental.parallel.ParallelMethodTest;
import org.junit.tests.experimental.rules.BlockJUnit4ClassRunnerOverrideTest;
import org.junit.tests.experimental.rules.ClassRulesTest;
-import org.junit.tests.experimental.rules.ExpectedExceptionRuleTest;
+import org.junit.tests.experimental.rules.ExpectedExceptionTest;
import org.junit.tests.experimental.rules.ExternalResourceRuleTest;
import org.junit.tests.experimental.rules.MethodRulesTest;
import org.junit.tests.experimental.rules.NameRulesTest;
@@ -144,7 +144,7 @@
ParentRunnerTest.class,
NameRulesTest.class,
ClassRulesTest.class,
- ExpectedExceptionRuleTest.class,
+ ExpectedExceptionTest.class,
TempFolderRuleTest.class,
ExternalResourceRuleTest.class,
VerifierRuleTest.class,
View
159 src/test/java/org/junit/tests/experimental/rules/EventCollector.java
@@ -0,0 +1,159 @@
+package org.junit.tests.experimental.rules;
+
+import static org.hamcrest.core.IsEqual.equalTo;
+import static org.junit.matchers.JUnitMatchers.both;
+
+import java.util.ArrayList;
+import java.util.List;
+
+import org.hamcrest.Matcher;
+import org.junit.internal.matchers.TypeSafeMatcher;
+import org.junit.runner.Description;
+import org.junit.runner.Result;
+import org.junit.runner.notification.Failure;
+import org.junit.runner.notification.RunListener;
+
+class EventCollector extends RunListener {
+ static Matcher<EventCollector> everyTestRunSuccessful() {
+ return both(hasNoFailure()).and(hasNoAssumptionFailure());
+ }
+
+ private static Matcher<EventCollector> hasNumberOfFailures(
+ final int numberOfFailures) {
+ return new TypeSafeMatcher<EventCollector>() {
+ @Override
+ public boolean matchesSafely(EventCollector item) {
+ return item.fFailures.size() == numberOfFailures;
+ }
+
+ public void describeTo(org.hamcrest.Description description) {
+ description.appendText("has ");
+ description.appendValue(numberOfFailures);
+ description.appendText(" failures");
+ }
+ };
+ }
+
+ static Matcher<EventCollector> hasSingleFailure() {
+ return hasNumberOfFailures(1);
+ }
+
+ static Matcher<EventCollector> hasNoFailure() {
+ return hasNumberOfFailures(0);
+ }
+
+ private static Matcher<EventCollector> hasNumberOfAssumptionFailures(
+ final int numberOfFailures) {
+ return new TypeSafeMatcher<EventCollector>() {
+ @Override
+ public boolean matchesSafely(EventCollector item) {
+ return item.fAssumptionFailures.size() == numberOfFailures;
+ }
+
+ public void describeTo(org.hamcrest.Description description) {
+ description.appendText("has ");
+ description.appendValue(numberOfFailures);
+ description.appendText(" assumption failures");
+ }
+ };
+ }
+
+ static Matcher<EventCollector> hasSingleAssumptionFailure() {
+ return hasNumberOfAssumptionFailures(1);
+ }
+
+ static Matcher<EventCollector> hasNoAssumptionFailure() {
+ return hasNumberOfAssumptionFailures(0);
+ }
+
+ static Matcher<EventCollector> hasSingleFailureWithMessage(String message) {
+ return hasSingleFailureWithMessage(equalTo(message));
+ }
+
+ static Matcher<EventCollector> hasSingleFailureWithMessage(
+ final Matcher<String> messageMatcher) {
+ return new TypeSafeMatcher<EventCollector>() {
+ @Override
+ public boolean matchesSafely(EventCollector item) {
+ return hasSingleFailure().matches(item)
+ && messageMatcher.matches(item.fFailures.get(0)
+ .getMessage());
+ }
+
+ public void describeTo(org.hamcrest.Description description) {
+ description.appendText("has single failure with message ");
+ messageMatcher.describeTo(description);
+ }
+ };
+ }
+
+ private final List<Description> fTestRunsStarted= new ArrayList<Description>();
+
+ private final List<Result> fTestRunsFinished= new ArrayList<Result>();
+
+ private final List<Description> fTestsStarted= new ArrayList<Description>();
+
+ private final List<Description> fTestsFinished= new ArrayList<Description>();
+
+ private final List<Failure> fFailures= new ArrayList<Failure>();
+
+ private final List<Failure> fAssumptionFailures= new ArrayList<Failure>();
+
+ private final List<Description> fTestsIgnored= new ArrayList<Description>();
+
+ @Override
+ public void testRunStarted(Description description) throws Exception {
+ fTestRunsStarted.add(description);
+ }
+
+ @Override
+ public void testRunFinished(Result result) throws Exception {
+ fTestRunsFinished.add(result);
+ }
+
+ @Override
+ public void testStarted(Description description) throws Exception {
+ fTestsStarted.add(description);
+ }
+
+ @Override
+ public void testFinished(Description description) throws Exception {
+ fTestsFinished.add(description);
+ }
+
+ @Override
+ public void testFailure(Failure failure) throws Exception {
+ fFailures.add(failure);
+ }
+
+ @Override
+ public void testAssumptionFailure(Failure failure) {
+ fAssumptionFailures.add(failure);
+ }
+
+ @Override
+ public void testIgnored(Description description) throws Exception {
+ fTestsIgnored.add(description);
+ }
+
+ @Override
+ public String toString() {
+ StringBuilder sb= new StringBuilder();
+ sb.append(fTestRunsStarted.size());
+ sb.append(" test runs started, ");
+ sb.append(fTestRunsFinished.size());
+ sb.append(" test runs finished, ");
+ sb.append(fTestsStarted.size());
+ sb.append(" tests started, ");
+ sb.append(fTestsFinished.size());
+ sb.append(" tests finished, ");
+ sb.append(fFailures.size());
+ sb.append(" failures, ");
+ sb.append(fAssumptionFailures.size());
+ sb.append(" assumption failures, ");
+ sb.append(fTestsIgnored.size());
+ sb.append(" tests ignored");
+
+ return sb.toString();
+ }
+}
View
232 src/test/java/org/junit/tests/experimental/rules/ExpectedExceptionRuleTest.java
@@ -1,232 +0,0 @@
-package org.junit.tests.experimental.rules;
-
-import static org.hamcrest.CoreMatchers.any;
-import static org.junit.Assert.assertThat;
-import static org.junit.experimental.results.PrintableResult.testResult;
-import static org.junit.experimental.results.ResultMatchers.hasSingleFailureContaining;
-import static org.junit.experimental.results.ResultMatchers.isSuccessful;
-import static org.junit.matchers.JUnitMatchers.both;
-import org.hamcrest.Description;
-import org.hamcrest.Matcher;
-import org.junit.Rule;
-import org.junit.Test;
-import org.junit.internal.matchers.TypeSafeMatcher;
-import org.junit.rules.ExpectedException;
-
-public class ExpectedExceptionRuleTest {
- public static class HasExpectedException {
- @Rule
- public ExpectedException thrown= ExpectedException.none();
-
- @Test
- public void throwsNothing() {
-
- }
-
- @Test
- public void throwsNullPointerException() {
- thrown.expect(NullPointerException.class);
- throw new NullPointerException();
- }
-
- @Test
- public void throwsNullPointerExceptionWithMessage() {
- thrown.expect(NullPointerException.class);
- thrown.expectMessage("happened?");
- throw new NullPointerException("What happened?");
- }
- }
-
- @Test
- public void expectedExceptionPasses() {
- assertThat(testResult(HasExpectedException.class), isSuccessful());
- }
-
- public static class HasWrongExpectedException {
- @Rule
- public ExpectedException thrown= ExpectedException.none();
-
- @Test
- public void throwsNullPointerException() {
- thrown.expect(NullPointerException.class);
- throw new IllegalArgumentException();
- }
- }
-
- @Test
- public void unExpectedExceptionFails() {
- assertThat(
- testResult(HasWrongExpectedException.class),
- hasSingleFailureContaining("Expected: an instance of java.lang.NullPointerException"));
- }
-
- public static class HasWrongMessage {
- @Rule
- public ExpectedException thrown= ExpectedException.none();
-
- @Test
- public void throwsNullPointerException() {
- thrown.expectMessage("expectedMessage");
- throw new IllegalArgumentException("actualMessage");
- }
- }
-
- @Test
- public void wrongMessageFails() {
- assertThat(
- testResult(HasWrongMessage.class), both(
- hasSingleFailureContaining("expectedMessage")).and(
- hasSingleFailureContaining("actualMessage")));
- }
-
- public static class WronglyExpectsException {
- @Rule
- public ExpectedException thrown= ExpectedException.none();
-
- @Test
- public void doesntThrowNullPointerException() {
- thrown.expect(NullPointerException.class);
- }
- }
-
- @Test
- public void failsIfExceptionNeverComes() {
- assertThat(
- testResult(WronglyExpectsException.class),
- hasSingleFailureContaining("Expected test to throw an instance of java.lang.NullPointerException"));
- }
-
- public static class WronglyExpectsExceptionMessage {
- @Rule
- public ExpectedException thrown= ExpectedException.none();
-
- @Test
- public void doesntThrowAnything() {
- thrown.expectMessage("anything!");
- }
- }
-
- @Test
- public void failsIfExceptionMessageNeverComes() {
- assertThat(
- testResult(WronglyExpectsExceptionMessage.class),
- hasSingleFailureContaining("Expected test to throw exception with message a string containing \"anything!\""));
- }
-
- public static class ExpectsSubstring {
- @Rule
- public ExpectedException thrown= ExpectedException.none();
-
- @Test
- public void throwsMore() {
- thrown.expectMessage("anything!");
- throw new NullPointerException(
- "This could throw anything! (as long as it has the right substring)");
- }
- }
-
- @Test
- public void passesWithSubstringMethod() {
- assertThat(testResult(ExpectsSubstring.class), isSuccessful());
- }
-
- public static class ExpectsSubstringNullMessage {
- @Rule
- public ExpectedException thrown= ExpectedException.none();
-
- @Test
- public void throwsMore() {
- thrown.expectMessage("anything!");
- throw new NullPointerException();
- }
- }
-
- @Test
- public void failsWithNullExceptionMessage() {
- assertThat(testResult(ExpectsSubstringNullMessage.class),
- hasSingleFailureContaining("NullPointerException"));
- }
-
- public static class ExpectsMessageMatcher {
- @Rule
- public ExpectedException thrown= ExpectedException.none();
-
- @Test
- public void throwsMore() {
- thrown.expectMessage(startsWith("Ack"));
- throw new NullPointerException("Ack!");
- }
- }
-
- @Test
- public void succeedsWithMessageMatcher() {
- assertThat(testResult(ExpectsMessageMatcher.class), isSuccessful());
- }
-
- public static class ExpectedMessageMatcherFails {
- @Rule
- public ExpectedException thrown= ExpectedException.none();
-
- @Test
- public void throwsMore() {
- thrown.expectMessage(startsWith("Wrong start"));
- throw new NullPointerException("Back!");
- }
- }
-
-
-
- private static Matcher<String> startsWith(final String prefix) {
- return new TypeSafeMatcher<String>() {
- public void describeTo(Description description) {
- description.appendText("starts with ");
- description.appendText(prefix);
- }
-
- @Override
- public boolean matchesSafely(String item) {
- return item.startsWith(prefix);
- }
- };
- }
-
- @Test
- public void failsWithMatcher() {
- assertThat(testResult(ExpectedMessageMatcherFails.class),
- hasSingleFailureContaining("Wrong start"));
- }
-
- public static class ExpectsMatcher {
- @Rule
- public ExpectedException thrown= ExpectedException.none();
-
- @Test
- public void throwsMore() {
- thrown.expect(any(Throwable.class));
- throw new NullPointerException("Ack!");
- }
- }
-
- @Test
- public void succeedsWithMatcher() {
- assertThat(testResult(ExpectsMatcher.class), isSuccessful());
- }
-
- public static class ExpectsMultipleMatchers {
- @Rule
- public ExpectedException thrown= ExpectedException.none();
-
- @Test
- public void throwsMore() {
- thrown.expect(IllegalArgumentException.class);
- thrown.expectMessage("Ack!");
- throw new NullPointerException("Ack!");
- }
- }
-
- @Test
- public void failsWithMultipleMatchers() {
- assertThat(testResult(ExpectsMultipleMatchers.class),
- hasSingleFailureContaining("IllegalArgumentException"));
- }
-}
View
324 src/test/java/org/junit/tests/experimental/rules/ExpectedExceptionTest.java
@@ -0,0 +1,324 @@
+package org.junit.tests.experimental.rules;
+
+import static java.util.Arrays.asList;
+import static org.hamcrest.CoreMatchers.any;
+import static org.hamcrest.core.IsEqual.equalTo;
+import static org.junit.Assert.assertThat;
+import static org.junit.Assert.fail;
+import static org.junit.Assume.assumeTrue;
+import static org.junit.rules.ExpectedException.none;
+import static org.junit.tests.experimental.rules.EventCollector.everyTestRunSuccessful;
+import static org.junit.tests.experimental.rules.EventCollector.hasSingleAssumptionFailure;
+import static org.junit.tests.experimental.rules.EventCollector.hasSingleFailure;
+import static org.junit.tests.experimental.rules.EventCollector.hasSingleFailureWithMessage;
+
+import java.io.IOException;
+import java.util.Collection;
+
+import org.hamcrest.Description;
+import org.hamcrest.Matcher;
+import org.junit.Rule;
+import org.junit.Test;
+import org.junit.internal.AssumptionViolatedException;
+import org.junit.internal.matchers.TypeSafeMatcher;
+import org.junit.rules.ExpectedException;
+import org.junit.runner.JUnitCore;
+import org.junit.runner.RunWith;
+import org.junit.runners.Parameterized;
+import org.junit.runners.Parameterized.Parameters;
+
+@RunWith(Parameterized.class)
+public class ExpectedExceptionTest {
+ private static final String ARBITRARY_MESSAGE= "arbitrary message";
+
+ @Parameters
+ public static Collection<Object[]> testsWithEventMatcher() {
+ return asList(new Object[][] {
+ { EmptyTestExpectingNoException.class, everyTestRunSuccessful() },
+ { ThrowExceptionWithExpectedType.class,
+ everyTestRunSuccessful() },
+ { ThrowExceptionWithExpectedPartOfMessage.class,
+ everyTestRunSuccessful() },
+ {
+ ThrowExceptionWithWrongType.class,
+ hasSingleFailureWithMessage(startsWith("\nExpected: an instance of java.lang.NullPointerException")) },
+ {
+ HasWrongMessage.class,
+ hasSingleFailureWithMessage("\nExpected: exception with message a string containing \"expectedMessage\"\n"
+ + " got: <java.lang.IllegalArgumentException: actualMessage>\n") },
+ {
+ ThrowNoExceptionButExpectExceptionWithType.class,
+ hasSingleFailureWithMessage("Expected test to throw an instance of java.lang.NullPointerException") },
+ { WronglyExpectsExceptionMessage.class, hasSingleFailure() },
+ { ExpectsSubstring.class, everyTestRunSuccessful() },
+ {
+ ExpectsSubstringNullMessage.class,
+ hasSingleFailureWithMessage(startsWith("\nExpected: exception with message a string containing \"anything!\"")) },
+ { ExpectsMessageMatcher.class, everyTestRunSuccessful() },
+ {
+ ExpectedMessageMatcherFails.class,
+ hasSingleFailureWithMessage(startsWith("\nExpected: exception with message \"Wrong start\"")) },
+ { ExpectsMatcher.class, everyTestRunSuccessful() },
+ { ThrowExpectedAssumptionViolatedException.class,
+ everyTestRunSuccessful() },
+ { ThrowAssumptionViolatedExceptionButExpectOtherType.class,
+ hasSingleFailure() },
+ { ViolateAssumptionAndExpectException.class,
+ hasSingleAssumptionFailure() },
+ { ThrowExpectedAssertionError.class, everyTestRunSuccessful() },
+ {
+ ThrowUnexpectedAssertionError.class,
+ hasSingleFailureWithMessage(startsWith("\nExpected: an instance of java.lang.NullPointerException")) },
+ { FailAndDontHandleAssertinErrors.class,
+ hasSingleFailureWithMessage(ARBITRARY_MESSAGE) },
+ {
+ ExpectsMultipleMatchers.class,
+ hasSingleFailureWithMessage(startsWith("\nExpected: (exception with message a string containing \"Ack!\" and an instance of java.lang.IllegalArgumentException)")) } });
+ }
+
+ private final Class<?> classUnderTest;
+
+ private final Matcher<EventCollector> matcher;
+
+ public ExpectedExceptionTest(Class<?> classUnderTest,
+ Matcher<EventCollector> matcher) {
+ this.classUnderTest= classUnderTest;
+ this.matcher= matcher;
+ }
+
+ @Test
+ public void runTestAndVerifyResult() {
+ EventCollector collector= new EventCollector();
+ JUnitCore core= new JUnitCore();
+ core.addListener(collector);
+ core.run(classUnderTest);
+ assertThat(collector, matcher);
+ }
+
+ public static class EmptyTestExpectingNoException {
@dsaff Owner
dsaff added a note

For years now, I've wanted to be able to write tests like this, but with a hitch: the matcher for results would be attached to the test class, like this:

public static class EmptyTestExpectingNoException {
public static Matcher<?> EXPECTED = everyTestRunSuccessful()

@Rule public ExpectedException thrown= none();
@Test public void throwsNothing() {}
}

Would you be willing and able to rewrite this test class in that style?

@stefanbirkner Collaborator

I give it a try.

@Tibor17
Tibor17 added a note

@David, Stefan

I think that we can the best if we use the Matcher.
I have reported IsThrowable matcher
https://github.com/KentBeck/junit/issues/322
which might be used.

I prefer a generic form rather than a concrete method names handleAssertionErrors(), handleAssumptionViolatedExceptions().

What about to use the rules like this


@Rule
public ExpectedException thrown= new ExpectedException(Matcher);

We can use anyOf() with these matcher. The only thing is to implement IBlock with run() method which throws anything.

@dsaff Owner
dsaff added a note

Tibor,

ExpectedException already has expect(Matcher). What are you proposing to use instead?

@stefanbirkner Collaborator

Finally I created the JUnitSelfTestRunner. See #361.

@dsaff Owner
dsaff added a note

Stefan,

Would you prefer we pull this change in unchanged, or should we finish #361, and use that for these tests here?

@stefanbirkner Collaborator

I would prefer to pull this change in unchanged.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
+ @Rule
+ public ExpectedException thrown= none();
+
+ @Test
+ public void throwsNothing() {
+ }
+ }
+
+ public static class ThrowExceptionWithExpectedType {
+ @Rule
+ public ExpectedException thrown= none();
+
+ @Test
+ public void throwsNullPointerException() {
+ thrown.expect(NullPointerException.class);
+ throw new NullPointerException();
+ }
+ }
+
+ public static class ThrowExceptionWithExpectedPartOfMessage {
+ @Rule
+ public ExpectedException thrown= none();
+
+ @Test
+ public void throwsNullPointerExceptionWithMessage() {
+ thrown.expect(NullPointerException.class);
+ thrown.expectMessage(ARBITRARY_MESSAGE);
+ throw new NullPointerException(ARBITRARY_MESSAGE + "something else");
+ }
+ }
+
+ public static class ThrowExceptionWithWrongType {
+ @Rule
+ public ExpectedException thrown= none();
+
+ @Test
+ public void throwsNullPointerException() {
+ thrown.expect(NullPointerException.class);
+ throw new IllegalArgumentException();
+ }
+ }
+
+ public static class HasWrongMessage {
+ @Rule
+ public ExpectedException thrown= none();
+
+ @Test
+ public void throwsNullPointerException() {
+ thrown.expectMessage("expectedMessage");
+ throw new IllegalArgumentException("actualMessage");
+ }
+ }
+
+ public static class ThrowNoExceptionButExpectExceptionWithType {
+ @Rule
+ public ExpectedException thrown= none();
+
+ @Test
+ public void doesntThrowNullPointerException() {
+ thrown.expect(NullPointerException.class);
+ }
+ }
+
+ public static class WronglyExpectsExceptionMessage {
+ @Rule
+ public ExpectedException thrown= none();
+
+ @Test
+ public void doesntThrowAnything() {
+ thrown.expectMessage("anything!");
+ }
+ }
+
+ public static class ExpectsSubstring {
+ @Rule
+ public ExpectedException thrown= none();
+
+ @Test
+ public void throwsMore() {
+ thrown.expectMessage("anything!");
+ throw new NullPointerException(
+ "This could throw anything! (as long as it has the right substring)");
+ }
+ }
+
+ public static class ExpectsSubstringNullMessage {
+ @Rule
+ public ExpectedException thrown= none();
+
+ @Test
+ public void throwsMore() {
+ thrown.expectMessage("anything!");
+ throw new NullPointerException();
+ }
+ }
+
+ public static class ExpectsMessageMatcher {
+ @Rule
+ public ExpectedException thrown= none();
+
+ @Test
+ public void throwsMore() {
+ thrown.expectMessage(startsWith(ARBITRARY_MESSAGE));
+ throw new NullPointerException(ARBITRARY_MESSAGE + "!");
+ }
+ }
+
+ public static class ExpectedMessageMatcherFails {
+ @Rule
+ public ExpectedException thrown= none();
+
+ @Test
+ public void throwsMore() {
+ thrown.expectMessage(equalTo("Wrong start"));
+ throw new NullPointerException("Back!");
+ }
+ }
+
+ public static class ExpectsMatcher {
+ @Rule
+ public ExpectedException thrown= none();
+
+ @Test
+ public void throwsMore() {
+ thrown.expect(any(IOException.class));
+ throw new NullPointerException("Ack!");
+ }
+ }
+
+ public static class ExpectsMultipleMatchers {
+ @Rule
+ public ExpectedException thrown= none();
+
+ @Test
+ public void throwsMore() {
+ thrown.expect(IllegalArgumentException.class);
+ thrown.expectMessage("Ack!");
+ throw new NullPointerException("Ack!");
+ }
+ }
+
+ public static class FailAndDontHandleAssertinErrors {
+ @Rule
+ public ExpectedException thrown= none();
+
+ @Test
+ public void violatedAssumption() {
+ thrown.expect(IllegalArgumentException.class);
+ fail(ARBITRARY_MESSAGE);
+ }
+ }
+
+ public static class ThrowUnexpectedAssertionError {
+ @Rule
+ public ExpectedException thrown= none();
+
+ @Test
+ public void wrongException() {
+ thrown.handleAssertionErrors();
+ thrown.expect(NullPointerException.class);
+ throw new AssertionError("the unexpected assertion error");
+ }
+ }
+
+ public static class ThrowExpectedAssertionError {
+ @Rule
+ public ExpectedException thrown= none();
+
+ @Test
+ public void wrongException() {
+ thrown.handleAssertionErrors();
+ thrown.expect(AssertionError.class);
+ throw new AssertionError("the expected assertion error");
+ }
+ }
+
+ public static class ViolateAssumptionAndExpectException {
+ @Rule
+ public ExpectedException thrown= none();
+
+ @Test
+ public void violatedAssumption() {
+ // expect an exception, which is not an AssumptionViolatedException
+ thrown.expect(NullPointerException.class);
+ assumeTrue(false);
+ }
+ }
+
+ public static class ThrowAssumptionViolatedExceptionButExpectOtherType {
+ @Rule
+ public ExpectedException thrown= none();
+
+ @Test
+ public void wrongException() {
+ thrown.handleAssumptionViolatedExceptions();
+ thrown.expect(NullPointerException.class);
+ throw new AssumptionViolatedException("");
+ }
+ }
+
+ public static class ThrowExpectedAssumptionViolatedException {
+ @Rule
+ public ExpectedException thrown= none();
+
+ @Test
+ public void throwExpectAssumptionViolatedException() {
+ thrown.handleAssumptionViolatedExceptions();
+ thrown.expect(AssumptionViolatedException.class);
+ throw new AssumptionViolatedException("");
+ }
+ }
+
+ private static Matcher<String> startsWith(final String prefix) {
+ return new TypeSafeMatcher<String>() {
+ public void describeTo(Description description) {
+ description.appendText("starts with ");
+ description.appendText(prefix);
+ }
+
+ @Override
+ public boolean matchesSafely(String item) {
+ return item.startsWith(prefix);
+ }
+ };
+ }
+}
Something went wrong with that request. Please try again.