More flexible Categories implementation (with suggested implementation)

[ahochsteger]

I would need the possibility to annotate test methods with different categories and then scan the whole classpath for matching tests:

/* MyTestSuite.java */
/** MyTestSuite runs all slow tests, excluding all test which require a network connection. */
@RunWith(FlexibleCategories.class)
@ExcludeCategory(OnlineTestCategory.class)
@IncludeCategory(SlowTestCategory.class)
@TestScanPackage("my.package")
@TestClassPrefix("")
@TestClassSuffix("Test")
public class MyTestSuite {
}

This would be an example test class:

/* SampleTest.java */
public class SampleTest {
    @Test
    @Category({OnlineTestCategory.class, SlowTestCategory.class})
    public void onlineAndSlowTestCategoryMethod() {
    }

    @Test
    @Category(OnlineTestCategory.class)
    public void onlineTestCategoryMethod() {
    }

    @Test
    @Category(SlowTestCategory.class)
    public void slowTestCategoryMethod() {
    }

    @Test
    public void noTestCategoryMethod() {
    }
}

On my blog I've described a solution that is working for me:
http://highstick.blogspot.com/2011/11/howto-categorize-junit-test-methods-and.html

[ahochsteger]

See here for a suggesteed solution:
http://highstick.blogspot.com/2011/11/howto-categorize-junit-test-methods-and.html

[dsaff]

Categories support has had some improvements in the last 2 years. I think that the current implementation of Categories, plus http://johanneslink.net/projects/cpsuite.jsp, could do what is asked here. Let me know what would be missing.

[shri046]

Are there any plans to bring the two together in near future? I have a similar need with my test suite containing 200+ test classes. Keeping up with the @SuiteClasses becomes a real hassle. As mentioned a combination of categories and cpsuite definitely works, but is there any specific reason as to why we need a dependency on a third party project to achieve this?

Just looking for some general roadmap/direction before I pull in the third party dependency in to our project.

[marcphilipp]

At the moment there are no concrete plans to bring the two together. Most IDEs and build tools have there own implementation for collecting test classes, though. We would like to provide something that renders those unnecessary. However, that's a big effort and we didn't get to it yet.