Parallel execution definitions of tests are ignored

[tam4s]

Play ignores SBT test configuration settings like parallelExecution in Test := true or test grouping, or defined limits like Tags.limit(Tags.Test, 3).

It's not clear (especially not for a Java developer) and undocumented how to execute tests parallel, even if it's possible somehow. Here is an example Build.scala that defines the tests running in multiple JVMs & multiple threads, but all the tests running in a single JVM in a single thread instead.

import sbt._
import Keys._
import play.Project._

object ApplicationBuild extends Build {
    val appName         = "testTest"
    val appVersion      = "1.0-SNAPSHOT"

    val testThreads = 8

    def groupByFirst(tests: Seq[TestDefinition]) =
      tests groupBy (_.name(0)) map {
        case (letter, tests) => new Tests.Group(letter.toString, tests, Tests.SubProcess(Seq("-Dfirst.letter"+letter)))
      } toSeq;

    val appDependencies = Seq(
      javaCore,
    )

    val main = play.Project(appName, appVersion, appDependencies).settings(
      concurrentRestrictions in Global := Seq(
        Tags.limit(Tags.ForkedTestGroup, testThreads),
        Tags.limit(Tags.Test, testThreads),
        Tags.limit(Tags.CPU, testThreads),
        Tags.limit(Tags.Compile, testThreads),
        Tags.limitAll(testThreads),
        Tags.limitUntagged(testThreads)),
      parallelExecution in Test := true,
      testGrouping <<= definedTests in Test map groupByFirst
    )
}

I use JUnit as my testing framework.

I put this line into each tests to test whether the tests run in parallel or not:

System.out.println(String.format("process id: %s thread id: %s", ManagementFactory.getRuntimeMXBean()
    .getName(), Thread.currentThread().getId()));

I understand that certain tests cannot be run parallel (E.g. more than one TestServer in a single JVM or more than one TestServer in multiple JVMs that listens on the same port) but it doesn't mean that parallel execution should be disabled for all kind of tests.

@Gissues:{"order":35.89743589743585,"status":"backlog"}

[jroper]

In Play 2.1.x, we introduced forking tests being enabled by default, this was to solve some classloader leaks we were having caused by third party libraries for people that run console for a long time and execute the tests many times. The SBT documentation on forked tests:

http://www.scala-sbt.org/0.12.3/docs/Detailed-Topics/Testing

Says that all tests in a particular fork group are run in serial, but you can configure multiple groups to fork multiple JVMs.

Alternatively, you can:

fork in Test := false,
parallelExecution in Test := true

And this will run the tests in the SBT JVM in parallel.

[nsphung]

It seems

fork in Test := false
parallelExecution in Test := true

parameters in build.sbt are ignored in Play 2.2.1

[axblueblader]

Hello have anyone got it to work, I've searched and tried all the settings still cannot configure parallel test execution