Testing classes don't support Parameterized tests?

[synhershko]

Was trying this:

@RunWith(Parameterized.class)
@ElasticsearchIntegrationTest.ClusterScope(scope = ElasticsearchIntegrationTest.Scope.TEST, numDataNodes = 1)
public class SandBoxTests extends ElasticsearchIntegrationTest {
//..
}

And got this error:

java.lang.IllegalStateException: No context information for thread: Thread[id=1, name=main, state=RUNNABLE, group=main]. Is this thread running under a class com.carrotsearch.randomizedtesting.RandomizedRunner runner context? Add @RunWith(class com.carrotsearch.randomizedtesting.RandomizedRunner.class) to your test class. Make sure your code accesses random contexts within @BeforeClass and @AfterClass boundary (for example, static test class initializers are not permitted to access random contexts).
    at com.carrotsearch.randomizedtesting.RandomizedContext.context(RandomizedContext.java:207)
    at com.carrotsearch.randomizedtesting.RandomizedContext.current(RandomizedContext.java:145)
    at org.apache.lucene.util.TestRuleSetupAndRestoreClassEnv.before(TestRuleSetupAndRestoreClassEnv.java:144)
    at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
    at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
    at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
    at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
    at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
    at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
    at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
    at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
    at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
    at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
    at org.junit.rules.RunRules.evaluate(RunRules.java:18)
    at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
    at org.junit.runner.JUnitCore.run(JUnitCore.java:157)
    at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:74)
    at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:211)
    at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:67)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at com.intellij.rt.execution.application.AppMain.main(AppMain.java:134)

/cc @tlrx

[synhershko]

Ok, that's a non-issue, but will be nice to have the work around documented:

Instead of using JUnit's Parameterized, one needs to add

@RunWith(com.carrotsearch.randomizedtesting.RandomizedRunner.class)

and then provide the parameters as

  @ParametersFactory
    public static Collection<Object[]> strategies() {

[javanna]

Exactly @synhershko as you found out, ElasticsearchIntegrationTest needs the RandomizedRunner runner. You can't extend it and use another runner. If you are interested in parametrized tests though, the randomized-testing project provides its own @ParametersFactory annotation that is compatibile with the randomized runner. We use it in ElasticsearchRestTests if you want to look at an example.

I don't see where we could document this to be honest, it's more something to be aware of since we use randomized-testing. Any ideas?

[synhershko]

Adding a new page here about parameterized tests with ES's testing suite would make sense: http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/integration-tests.html

That whole section is kinda lacking - some examples as well as current version numbers in http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/using-elasticsearch-test-classes.html could greatly help