I would like to be able to hook in before and after each of the individual test runs in a theory. This is so I can rollback database transactions between each test run. I can use the junit before and after test but that is not fine grained enough as it is run before and after each theory.

Also I'm using theories with spring integration tests (AbstractTransactionalDataSourceSpringContextTests). So my test class is extending AbstractTransactionalDataSourceSpringContextTests and using the theories test runner. It would be nice to have a Spring theories runner to make it easier to hook up theories with spring Integration tests. This is how I came across this issue. But being able to hook into before and after each theory test is a nice start.

If have a simple pojo as a datapoint and then in my test I set one of the member variables. Every subsequent test will then have that member variable set.

It seems like the datapoints are being created after the class is setup. I think the datapoints should be setup before each theory to stop this issue.

I have not be able to find a way for theories to automatically generation datapoints for me. I have read about a agitar runner but I cant find much doco about it. Atm I have been creating data points or using the parameter supplier annotation. But it would be nice for theories to have a data generator out of the box. Could the datapoints generation be smart like how PEX(http://research.microsoft.com/en-us/projects/Pex/) can find boundary conditions?

What a mean is when a test needs actual and expected values to test with. For example what if you want to test a dollar to text converter. So take $10.00 the answer would be "ten dollars and zero cents" for instance. It would be impossible to pass in $10.00 and verifiy that. How would we test this using theories? I got this example from a friends blog see here http://dibblego.wordpress.com/2009/05/30/twelve-vigintillion-dollars-and-one-cent/

It is not included in the distribution, and does not seem to be available online anywhere. Anybody hava a pointer?

The Maven metadata on Central is incomplete for junit/junit, and non-existent for junit/junit-dep, so one cannot use version ranges when specifying dependencies on junit. Also, I think this also means the default version is an old version (4.4).

I'm not sure who takes care of uploading to Maven, but perhaps this could be looked at or passed along. Thanks!

The underlying request of this issue is to run a limited amount of tests in
a concurrent manner, something that is often required when back-end systems are involved. With very large test-sets there is also a need to throttle the amount of
tests being executed in parallel in a rational manner, a desired scenario is parallel=classes, but with a limited amount of threads (= limited number of concurrent classes).

The current implementation of ParallelComputer is also slower than Computer for all use cases I have tested. The reasons for this are as follows:
A) Each suite gets a separate thread-pool. This pool is not shut down at the end of the suite, so the threads linger around for 60 seconds until the pool decides to terminate. If you're running 100 test classes, you can easily have several thousand threads hanging around at the end of the test-run. This is not good.
B) Termination conditions are a maze of threads; when running with classes=methods=true, you basically end up with a separate thread for the class that's waiting for the method threads to finish.
C) The design does not permit running with fixed-size thread-pools (at least not for methods=classes=true) , since there is potential for deadlocks. Fixed size thread-pools are important when you want to avoid bashing back-end systems in integration tests and webtests.

You may also want to have a look at my blog post on this subject:
http://incodewetrustinc.blogspot.com/2009/07/run-your-junit-tests-in-parallel-with.html

I have an alternate implementation which you're free to take whatever you like from ;) This implementation is not totally without issues either, especially around shutdown of the computer (I believe the computer should be responsible for determining if all the worker threads are finished)

git://github.com/krosenvold/configurable-parallel-computer.git

http://support.github.com/discussions/repos/1174-problem-with-fetching-tags

I'm an avid user of hamcrest, and it's really confusing for other developers when they get

java.lang.NoSuchMethodError: org.hamcrest.Matcher.describeMismatch(Ljava/lang/Object;Lorg/hamcrest/Description;)V

or some other error because we use a later version of hamcrest. It's usually a simple classpath ordering issue, but it can get in the way a bit.

As we are on a later version, we also use MatcherAssert.assertThat instead of Assert.assertThat (as it gives/gave much better error messages anyhow).

What I would like is just to delete the hamcrest classes after the build. If we have a compatible version of hamcrest for Assert.assertThat() etc, then that will work, if not, then we know what to do anyhow. (as it will show the hamcrest classes. It's quite confusing for a developer the first time they see hamcrest inside junit :))

Cheers

My question now is: is there something like a @BeforeClass
interceptor? Obviously, this is for the same reason we might need
@BeforeClass in the first place—to provide an interceptor that performs potentially expensive setup/teardown before/after all @Test
methods.

ExpectedException javadocs show use of private constructor - should be .none()?

I really don't see why fields with a @Rule annotations must be public and it's quite easy to fix...

Merely voicing out an idea here that came to mind as I've been using @Rules. I'd like to put it in here just for the record, but if you guys have a better idea for the direction @Rules should go I'd be happy to go with that.

Currently, to use a rule the test class has to declare and instantiate the Rule object, like so:

    @Rule
    public MagicMocker magicMocker = new MagicMocker();

Basically, I can see myself, and others, coming up with quite a few rules that the test itself doesn't interact with (and that have default constructors).

My idea is to let the test simply declare any such rules, using another annotation, and have JUnit take care of instantiation.

JUnit already instantiates the test classes themselves, so I don't see why it can't instantiate other things. JUnit also already provides class-level annotations that affect test behavior (@RunWith) so I think whole the idea isn't entirely foreign to JUnit.

For example,

@Rules({TestPersistenceContext.class, FixturesLoader.class})
public class DaoTest {

In the above, JUnit could take care of instantiating, then applying the rules that set up the test database and load data fixtures. The FixturesLoader.class could be a MethodRule that gets applied to all test methods. The resulting test code is more concise and less cluttered with rules/fixture setup.

A further refinement would be to allow @Rules to selectively applied to methods. A contrived example:

    @Test
    @Rules(ExpectedSqlException.class)
    public void shouldThrowSqlException() {
        ....

In the above case, the ExpectedSqlException rule will only be instantiated and applied to that particular test method. All other test methods will only have the @Rules declared at the class-level.

I realize I'm subverting Rules and using them as fixtures/scaffolding, but that's a discussion worthy of its own thread.

I also realize my above use-cases can be solved by using a custom Runner, but in my mind a Runner represents the run-time context of a test and thus tests shouldn't have to care about who runs them. Plus, you can't use multiple Runners, can you?

What do you guys think?

I see versions 4.6 and 4.7 of the bundled-libraries version of JUnit in the Maven Central repo, but no junit-dep. Can this be added by whoever is doing the uploads?

Hi!

I would have a suggestion (acctually this is mostly
implemented and works well for me) for an extension to the
way tests are added to Suites in JUnit: the suite gets two
new annotations: two regular expressions, which are matched
against class-files in the CLASSPATH. If the first regex
matches a class name, the class is included, except if the
second one matched also. Example:

@RunWith(Suite .class) @Suite .SuiteClasses(value = {}, include = ".Rational.Test", exclude = ".Black.") public class SomeTests { }

The modification is actually a rather simple modification
to class Suite (see below).

Maybe you consider to include this into the next release? If
yes, I would be willing to invest a little more time to sort
out issues that may come up.

I am aware that something like this can be done using
maven2 (surefire-plugin), this is fairly easy in ant, and
David Saff pointed me to a similar ides realised by
Johannes Link in ClasspathSuite. However, at least the first
two solutions don't work too well with eclipse, and the latter
requires yet another software package for a fairly simple
functionality.

Regards,

  Georg

import java.io.File;
import java.lang.annotation.ElementType;
import java.lang.annotation.Inherited;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.lang.annotation.Target;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.HashSet;
import java.util.List;
import java.util.Set;
import java.util.regex.Pattern;

import org.junit.internal.builders.AllDefaultPossibilitiesBuilder;
import org.junit.runner.Description;
import org.junit.runner.Runner;
import org.junit.runner.notification.RunNotifier;
import org.junit.runners.ParentRunner;
import org.junit.runners.model.InitializationError;
import org.junit.runners.model.RunnerBuilder;

/* Using Suite as a runner allows you to manually build a suite containing tests from many classes. It is the JUnit 4 equivalent of the JUnit 3.8.x static {@link junit.framework.Test} suite() method. To use it, annotate a class with @RunWith(Suite.class) and @SuiteClasses([{TestClass1.class, ...}][,include="regexp"[,exclude="regexp"]]) . When you run this class, it will run all specified tests classes. The following rules apply:

• The explicitly in the attribute {@code value} specified classes (if any) are always run.
• Any file in the class path (imported or not) having the postfix {@code .class} with its full class name matching the regular expression in the attribute {@code include} is included in the test, except if its class name also matches the regular expression in the attribute {@code exclude}.
• No effort is taken to exclude files which are not JUnit tests (or not even java class files).
• The regular expression must match the entire class name. E.g. the class {@code my.example.exampleTest}, matches the following: "{@code .Test}", " {@code .example.}", or "{@code .}, but not "{@code Test}".
• Classes stored in jar-files are disregarded.

    extends ParentRunner<Runner>

{

/**
 * Returns an empty suite.
 */
public static Runner emptySuite()
{
    try {
        return new Suite((Class<?>) null, new Class<?>[0]);
    } catch (InitializationError e) {
        throw new RuntimeException("This shouldn't be possible");
    }
}

/**
 * The <code>SuiteClasses</code> annotation specifies the classes to be run
 * when a class
 * annotated with <code>@RunWith(Suite.class)</code> is run.
 */
@Retention(RetentionPolicy.RUNTIME)
@Target(ElementType.TYPE)
@Inherited
public @interface SuiteClasses {
    /**
     * @return the classes to be run irrespective of the settings of {@code
     *         #include()} and {@code exclude}
     */
    public Class<?>[] value() default {};

    /**
     * @return the regular expression a class name has to match in order to
     *         be included into the test. Classes which match {@code
     *         include} <em>AND</em> {@code exclude} are <em>NOT</em>
     *         included.
     */
    public String include() default "";

    /**
     * @return the regular expression a class name may not match.
     */
    public String exclude() default "";
}

static Class<?>[] getAnnotatedClasses(Class<?> klass)
        throws InitializationError
{
    SuiteClasses annotation = klass.getAnnotation(SuiteClasses.class);
    if (annotation == null)
        throw new InitializationError(String.format(
                "class '%s' must have a SuiteClasses annotation", klass
                        .getName()));
    Pattern includePattern = Pattern.compile(annotation.include());
    Pattern excludePattern = Pattern.compile(annotation.exclude());
    if (annotation.include() != null && annotation.include().length() > 0) {
        List<String> paths = Arrays.asList(System.getProperty(
                "java.class.path").split(":"));
        HashSet<String> names = new HashSet<String>();
        for (String p : paths) {
            File file = new File(p);
            String testClass = isTestClass(file, "", includePattern,
                    excludePattern);
            if (testClass != null)
                names.add(testClass);
            else {
                if (file.isDirectory()) {
                    collectClassNames(file, "", includePattern,
                            excludePattern, names);
                }
            }
        }
        // System.err.println("Tests matching the pattern: " + names);
        ArrayList<Class<?>> classes = new ArrayList<Class<?>>(Arrays
                .asList(annotation.value()));
        for (String name : names) {
            try {
                classes.add(ClassLoader.getSystemClassLoader().loadClass(
                        name));
            } catch (ClassNotFoundException e) {
                System.err.println("Could not load class " + name);
            }
        }
        Class<?>[] classes_ = new Class[classes.size()];
        classes.toArray(classes_);
        return classes_;
    }
    return annotation.value();
}

/**
 * Tests file names for whether they represent a test
 * 
 * @param file
 *            a potential class-file
 * @param packageName
 *            The name of the package the {@code file} would be part of.
 * @param include
 *            the pattern the class name must match to be included.
 * @param exclude
 *            the pattern excluding classes.
 * @return the equivalent class name if the file can be assumed to be a
 *         class file (i.e. is a plain file
 *         and has a postfix {@code .class} and the class name matches
 *         {@link SuiteClasses#include()} but not
 *         {@link SuiteClasses#exclude()}.
 */
static String isTestClass(File file, String packageName, Pattern include,
        Pattern exclude)
{
    if (!file.isFile() || !file.getName().matches(".*\\.class"))
        return null;
    String klassName = (packageName.length() > 0) ? (packageName + "." + file
            .getName().replace(".class", ""))
            : file.getName().replace(".class", "");
    if (include.matcher(klassName).matches()
            && !exclude.matcher(klassName).matches()) return klassName;
    return null;

}

/**
 * search for classes with names matching the given pattern.
 * 
 * @param dir
 *            the directory which is recursively searched.
 * @param packageName
 *            the current package name (as based on the search through the
 *            deirctory hierarchy
 * @param includePattern
 *            a regular expression matched against fully quallified class
 *            names.
 * @param excludePattern
 *            The pattern used to exclude classes.
 * @param names
 *            the collection of found class names.
 */
private static void collectClassNames(final File dir,
        final String packageName, final Pattern includePattern,
        Pattern excludePattern, final Set<String> names)
{
    for (File file : dir.listFiles()) {
        String testClass = isTestClass(file, packageName, includePattern,
                excludePattern);
        if (testClass != null) {
            names.add(testClass);
        }
        else {
            if (file.isDirectory()) {
                collectClassNames(
                        file,
                        (packageName.length() > 0) ? (packageName + "." + file
                                .getName())
                                : file.getName(), includePattern,
                        excludePattern, names);
            }
        }
    }
}

private final List<Runner> fRunners;

/**
 * Called reflectively on classes annotated with
 * <code>@RunWith(Suite.class)</code>
 * 
 * @param klass
 *            the root class
 * @param builder
 *            builds runners for classes in the suite
 * @throws InitializationError
 */
public Suite(Class<?> klass, RunnerBuilder builder)
        throws InitializationError
{
    this(builder, klass, getAnnotatedClasses(klass));
}

/**
 * Call this when there is no single root class (for example, multiple class
 * names
 * passed on the command line to {@link org.junit.runner.JUnitCore}
 * 
 * @param builder
 *            builds runners for classes in the suite
 * @param classes
 *            the classes in the suite
 * @throws InitializationError
 */
public Suite(RunnerBuilder builder, Class<?>[] classes)
        throws InitializationError
{
    this(null, builder.runners(null, classes));
}

/**
 * Call this when the default builder is good enough. Left in for
 * compatibility with JUnit 4.4.
 * 
 * @param klass
 *            the root of the suite
 * @param suiteClasses
 *            the classes in the suite
 * @throws InitializationError
 */
protected Suite(Class<?> klass, Class<?>[] suiteClasses)
        throws InitializationError
{
    this(new AllDefaultPossibilitiesBuilder(true), klass, suiteClasses);
}

/**
 * Called by this class and subclasses once the classes making up the suite
 * have been determined
 * 
 * @param builder
 *            builds runners for classes in the suite
 * @param klass
 *            the root of the suite
 * @param suiteClasses
 *            the classes in the suite
 * @throws InitializationError
 */
protected Suite(RunnerBuilder builder, Class<?> klass,
        Class<?>[] suiteClasses) throws InitializationError
{
    this(klass, builder.runners(klass, suiteClasses));
}

/**
 * Called by this class and subclasses once the runners making up the suite
 * have been determined
 * 
 * @param klass
 *            root of the suite
 * @param runners
 *            for each class in the suite, a {@link Runner}
 * @throws InitializationError
 */
protected Suite(Class<?> klass, List<Runner> runners)
        throws InitializationError
{
    super(klass);
    fRunners = runners;
}

@Override
protected List<Runner> getChildren()
{
    return fRunners;
}

@Override
protected Description describeChild(Runner child)
{
    return child.getDescription();
}

@Override
protected void runChild(Runner runner, final RunNotifier notifier)
{
    runner.run(notifier);
}

}

(I was not sure how to formulate the summary...) This issue is also related to these JUnit bugs and is in version 4.7 (release), but may have entered already after version 4.4:

• "CompositeRunner.filter incorrect if child throws NoTestsRema" - ID: 1815188

• "Sorters and Filters are ignored in 4.5" - ID: 2008607

When using a filter on a composite test, i.e. ParentRunner and below, the filtering happens in two steps:
1. Partial filtering on composite test based on Filter class and Filterable interface 2. Full filtering when running the tests via ParentRunner class:
childrenInvoker(), runChildren() and getFilteredChildren() methods

I think the behaviour of JUnit 4.3.1 and 4.4 was different and better in that the filtering was done in a single step.
The problems of the current approach are in my mind:
The first filtering that throws the NoTestsRemainException is not complete, so it may happen that only the second filtering removes "silently" some more tests. This leads in my case to JUnit reports (via ant-junit) that contains test classes without test cases.
Performance could be improved: as filtering is done multiple times and shouldRun() is additionally used vai getDescription() and sorting in
getFilteredChildren())

As I think it was more or less suggested in one of the referenced bug reports above a quick "fix" (in fact a refactoring of the two step filtering should be strived for in my mind) is in ParentRunner.filter(Filter filter)
method:

Old:

public void filter(Filter filter) throws NoTestsRemainException {
    fFilter= filter;

    for (T each : getChildren())
        if (shouldRun(each))
            return;
    throw new NoTestsRemainException();
}

New:

public void filter(Filter filter) throws NoTestsRemainException {
    fFilter= filter;

    ArrayList<T> filtered= new ArrayList<T>();
    for (T each : getChildren()) {
        if (shouldRun(each)) {
            try {
                filterChild(each);
                filtered.add(each);
            } catch (NoTestsRemainException e) {
                // don't add it
            }
        }
    }
    if (filtered.isEmpty()) {
        throw new NoTestsRemainException();
    }
}

In the course of upgrading from JUnit 4.4 to JUnit 4.7 I stumbled over this
bug:

@Test
public void testMultipleFilters() throws Exception {
    JUnitCore junitCore = new JUnitCore();
    Request request = Request.aClass(ExampleTest.class);
    Request requestFiltered = request.filterWith(new SingleMethodNameFilter("test1"));
    Request requestFilteredFiltered = requestFiltered.filterWith(new SingleMethodNameFilter("test2"));
    Result result = junitCore.run(requestFilteredFiltered);
    printResult(result);
    assertEquals(1, result.getRunCount()); // Fails here: actual 2
}

private static class SingleMethodNameFilter extends Filter {

    private String methodName;

    public SingleMethodNameFilter(String methodName) {
        this.methodName = methodName;
    }

    @Override
    public boolean shouldRun(Description description) {
        return !description.getMethodName().equals(methodName);
    }

    @Override
    public String describe() {
        return "filter method name: " + methodName;
    }

}

public class ExampleTest {

@Test
public void test1() throws Exception {
    System.out.println("Run: test1");
    assertEquals(1, 1);
}

@Test
public void test2() throws Exception {
    System.out.println("Run: test2");
    assertEquals(1, 1);
}

@Test
public void test3() throws Exception {
    System.out.println("Run: test3");
    assertEquals(1, 1);
}

}

Of course one could make a composite Filter, but the old behaviour and the API seem to show that this limitation is not on purpose.

The Bug "ID: 2094316" ("Request.filterWith has bizarre behavior") is kind of related, but does not point to the main problem in my mind.

Moreover, the filtering is carried out too often in my mind (we are using filters for a range of QA/TA related aspects): e.g. in the
testSingleFilter() method the filters shouldRun() method is called 11 times for ExampleTest (containing 3 test cases):

@Test
public void testSingleFilter() throws Exception {
    Request requestFiltered = request.filterWith(new SingleMethodNameFilter("test1"));
    Result result = junitCore.run(requestFiltered);
    printResult(result);
    assertEquals(2, result.getRunCount());
}

Rules is a nice mechanism for composing rather than inheriting functionality.

This concept could be extended to suites as well, like the @BeforeClass/@AfterClass annotation on a suite.

Also, by being able to declare suite rules, test case rules and test rules on a parent suite level and then propagate them down in the test hierarchy, further DRYness can be achieved.

I have made a test implementation for this, available at
http://github.com/brolund/junit

This issue is related to issue 29

Add an annotation to the Suite class called SuiteMethod that would annotate
a method that returns a Class[] array. The Suite would then be built from
the two annotations, Suite.SuiteClasses plus any methods on the class (and
enforce that they be public, static, take no parameters and return a
Class[] array) that are annotated with Suite.SuiteMethod and build the
Suite as a combination of all of the Arrays returned. Multiple methods
return the same class, my preference would be to only run the class once.

An alternative would be to create a separate runner for a class has the
@SuiteMethod.

Hi!

If the suite InnerTestClass (see below) is run, an error stating "Test class should have exactly one public zero-argument constructor" is issued. However, the real issue is that the inner test class should be "static" (at least as long as JUnit does not provide a way to instantiate from an outer class).

From what I gather is, that a judiciously placed test like
if (clazz.isMemberClass() && ! Modifier.isStatic(clazz.getModifiers())

 throw new Error("inner class " + clazz.getName() + " must be declared static.");

should do the trick.

Cheers,
Georg

@RunWith(Suite.class) @SuiteClasses( {

InnerTestClass.T1.class

}) public class InnerTestClass
{

public class T1
{
    @Test public void test() {  }
}

}

We use a custom ParentRunner to run tests that are specified via custom xml
files for our product. We would greatly like to be able to encode the file
and line number of the underlying xml for a test in such a way that we can click on
the tests in the eclipse runner and jump straight to the xml file and the
line number.

Right now the test names are named sequentially 0,1, etc in Parameterized:

@Override protected String getName() {
return String.format("[%s]", fParameterSetNumber);
}

@Override protected String testName(final Method method) {
return String.format("%s[%s]", method.getName(), fParameterSetNumber);
}

It would help if the data themselves were to be used in determining the name. Proposed alternatives for name (which has to be determinited statically before Test is instantiated):

• call toString on first parameter
• call the a newly added names attribute to Parameters annotation (with default empty names for compatibility) public String[] names() default {};
• call a public, static (!) method annotated by newly-defined @TestName and accepting the same arguments as the constructor

I can provide (again) a patch. This issue was tracked as http://sourceforge.net/tracker/?func=detail&atid=365278&aid=1742040&group_id=15278 and https://sourceforge.net/tracker/?func=detail&atid=365278&aid=1630834&group_id=15278
I can submit a patch if needed

Currently there is no simple way for a method to be invoked before or after a Parameterized instance is ran.

Said another way, along with @BeforeClass and @AfterClass there should be @BeforeParameterRun @AfterParameterRun. Each fires before/after a Parameterized instance is ran.

[This ticket was copied from the old sourceforge tracker, original issue created Feb 21 2008] Currently, the only way to stop tests that are being run using JUnitCore,
is to implement a Runner that then has access to RunNotifier.pleaseStop().
This was possible to do in JUnit3 using TestResult.stop().

In my work place we found this feature very useful, as had programs that
simply ran tests and processed the output. These programs weren't too
coupled with JUnit, they simply listened to TestResult and processed the
data. We would like to do the same thing with Result and RunListener.

My suggestion is simply to add pleaseStop() to JUnitCore (which is always
used for simply running tests).

Thanks,
Aviv.

With or without cygwin:

c:\src\junitkb\build.xml:113: Execute failed: java.io.IOException: Cannot run program "build\Markdown.pl": CreateProcess error=193, %1 is not a valid Win32 application

Much as I dislike windows, I was hoping to build ;)

My kludge to build was to modify build.xml as follows:

<target name="release-notes" if="os.family!='windows' ">

[copied, on request, from http://sourceforge.net/tracker/?func=detail&atid=365278&aid=1941834&group_id=15278]

Please support invocationCount as an additional Parameter for the @Test-Annotation.

I think this would be easy and useful.

Additionally it would be useful to add successPercentage too.

(like TestNG)

Was implementing TestWatchman and realised that the class is not abstract but yet does nothing when used.
Both TestWatchman and Verifier should be made abstract like ExternalResource, to clearly document that they are meant to be extended and not to be used as it is.

like https://sourceforge.net/tracker/?func=detail&aid=1748683&group_id=15278&atid=365278

DataPoints declared in MultiPointsMethod will be matched incorrectly with array causing an exception due to argument type mismatch. Below is an example showing the bug:
@RunWith(Theories.class) public class TestDataPoints {

@DataPoint
public static String[] stringArr1= { "Good", "Morning" };

@DataPoints
public static String[] stringArr2() {
    return new String[] { "Hello", "World" };
}

/**
 * An array is expected so only stringArr1 should be passed in but
 * stringArr2 will be passed in as parameter,causing an exception due to
 * argument type mismatch
 * 
 * @param arr
 */
@Theory
public void testTheory(String[] arr) {

    for (int i= 0; i < arr.length; i++) {
        System.out.print(arr[i] + " ");
    }
}

}

DataPoints declared in MultiPointsMethod will be matched incorrectly with array causing an exception due to argument type mismatch. Below is an example showing the bug:

@RunWith(Theories.class)
public class TestDataPoints {

@DataPoint
public static String[] stringArr1=new String[]{new String("Good"),new String("Morning")};

@DataPoints
public static String[] stringArr2() {
    return new String[] {
        new String("Hello"),
        new String("World")
    };
}

/**
 * An array is expected so only stringArr1 should be passed in
 * but stringArr2 will be passed in as parameter,causing an 
 * exception due to argument type mismatch 
 * @param arr
 */
@Theory
public void testTheory(String[] arr) {

    for(int i=0;i<arr.length;i++)
    {
        System.out.print(arr[i]+" ");
    }
}

}

If you assumeThat(3, is(4)) your test will pass; instead, the test should be ignored.

Today's behavior is tested in AssumptionTest#failedAssumptionsMeanPassing, where it asserts that the test passes and is NOT ignored.

I'm explaining the issue with a unit test:

public class TemporaryFolderTest {

@Rule
public TemporaryFolder temporaryFolder = new TemporaryFolder();

@Before
public void before(){
    System.out.println(temporaryFolder.getRoot());
    // fails
    // assertNotNull(temporaryFolder.getRoot());
}

@Test
public void testTempFolder(){
    System.out.println(temporaryFolder.getRoot());
    assertNotNull(temporaryFolder.getRoot());
}

}

If the java "ext" directory is specified on the command line, JUnit is unable to find the unit test classes. Whereas, if "classpath" is used, everything is fine. When tests depend on a number of 3rd party libraries, the "ext" directory approach is far preferable.

For example:

java -classpath /home/cowan/src/atst/lib/java/classes
-Djava.ext.dirs=/home/cowan/src/atst/lib/java/ext org.junit.runner.JUnitCore atst.cs.test.TestAll

results in:
JUnit version 4.7
Could not find class: atst.cs.test.TestAll

The strange thing is that if the verbose flag is used, Java even logs that it successfully loaded the TestAll class from the appropriate directory before later displaying the error.

If the command line is modified to:

java -classpath
/home/cowan/src/atst/lib/java/classes:/home/cowan/src/atst/lib/java/ext/junit.jar org.junit.runner.JUnitCore atst.cs.test.TestAll

everything works as expected.

-Djava.ext.dirs seems to be causing classpath/classloader issues.

Theories runner warn if @DataPoint targets an instance field (as opposed to a static field) with an Error ("DataPoint field f must be static).

However, if @DataPoint targets a non-public field, the field is ignored silently. In the following test junit runner ignores ignoredSilently nonpublic static field:

@RunWith(Theories.class)
public class TestTheory1 {
    @DataPoint static String ignoredSilently = null;
    @DataPoint public static String field = "m";
    @Theory public void nonNull(String s) { Assert.assertNotNull(s); }
}

When generating parameter lists from field DataPoints, only the raw type gets considered, not the generic type.

Consider the following failing test:

@RunWith(Theories.class)
public class TheoryGenerics {
    @DataPoint public static List<String> strings = Arrays.asList("what");
    @DataPoint public static List<Integer> ints = Arrays.asList(1);

    @Theory
    public void regex(List<String> strings, List<Integer> ints) {
        Assert.assertThat(strings.get(0), Is.is(String.class));
        Assert.assertThat(ints.get(0), Is.is(Integer.class));
    }
}

It'll be great to be able to filter through possible data points by name, in addition to the Java type. String types are commonly used for various purposes.

Consider the following use case.

@RunWith(Theories.class)
public class DataPointNames {
    @DataPoint public static String regex1 = ".*";
    @DataPoint public static String sample1 = "(";

    @Theory
    public void consistentMatcher(String regex, String input) {
        assumeThat(input, isRegex());
        boolean a = input.matches(regex);
        boolean b = Pattern.matches(regex, input);
        Assert.assertEquals(a, b);
    }
}

It'll be great if we can express it as:

@RunWith(Theories.class)
public class DataPointNames {
    @DataPoint("regex") public static String regex1 = ".*";
    @DataPoint("input") public static String sample1 = "(";

    @Theory
    public void consistentMatcher(
            @DataPoint("regex") String regex,
            @DataPoint("input") String input) {
        boolean a = input.matches(regex);
        boolean b = Pattern.matches(regex, input);
        Assert.assertEquals(a, b);
    }
}

The JUnit javadocs are missing the @since entries that tell when an API is added.
This is especially important when viewing the javadocs on the web; you need to know
which minimum version of JUnit to request with Maven to get the API you're looking at.

"Global rules" are rules that are declared in one place and applied to all test cases/tests/suites in a test run.

"One place" could be a properties file, named by convention, e.g. "/global-junit-rules.properties", or they could be declared in a test suite and propagated (see http://github.com/KentBeck/junit/issues#issue/40), or in some other way.

From forum thread
http://tech.groups.yahoo.com/group/junit/message/22133

See conversation on http://github.com/KentBeck/junit/issues/#issue/69