TestKit Plugins

Provides a simple, opinionated structure for writing TestKit-based tests for Gradle plugins and collecting code coverage from them. Contains the following components:

• The main plugin, com.toasttab.testkit is applied to the plugin project.
• The junit5 extension injects the test project model into JUnit 5 tests.
• The coverage plugin, com.toasttab.testkit.coverage is applied to TestKit fixture projects.

Setup

In the plugin project, apply the main plugin and bring in the junit5 extension dependency.

plugins {
    kotlin("jvm")
    jacoco
    id("com.toasttab.testkit") version <<version>>
}

dependencies {
    testImplementation("com.toasttab.gradle.testkit:junit5:<<version>>")
}

Each test method is expected to have a corresponding test project located at src/test/projects/<<Test>>/<<method>>.

src/test/projects/MyTest/sometest:
   build.gradle.kts
   settings.gradle.kts # optional, but IntelliJ will complain

In the test project's build.gradle.kts, make sure to apply the coverage plugin.

plugins {
    id("com.toasttab.testkit.coverage")
}

Now, write the actual test. Note that a TestProject instance will be automatically injected into the test method.

@TestKit
class MyTest {
    @Test
    fun sometest(project: TestProject) {
        project.createRunner()
            .withArguments("check")
            .build()
    }
}

Parameterized Gradle versions

To run a test against multiple versions of Gradle, use the @ParameterizedWithGradleVersions annotation. Gradle versions can be specified per class in the @TestKit annotation or per method in the @ParameterizedWithGradleVersions annotation. Each gradle version argument will be automatically injected into the runner created via TestProject.createRunner.

@TestKit(gradleVersions = ["8.6", "8.7"])
class ParameterizedTest {
    @ParameterizedWithGradleVersions
    fun sometest(project: TestProject) {
        project.createRunner()
            .withArguments("check")
            .build() 
    }
}

Code coverage

It is notoriously difficult to collect code coverage data from TestKit tests. By default, TestKit tests launch in a separate Gradle daemon JVM, which lingers after the tests finish. Gradle attaches an agent to the daemon JVM which instruments all classes from the plugin classpath and makes it impossible for the Jacoco agent to instrument classes on the fly.

• The main plugin pre-instruments the plugin classes and other project classes that the plugin under test depends on using Jacoco offline instrumentation.
• The junit5 extension configures the TestKit build to use pre-instrumented classes, starts a Jacoco coverage TCP server, and points the jacoco runtime to the TCP server.
• The TCP server writes coverage data into a separate file and stops writing the file when the tests finish running. This allows the main Gradle process to collect task outputs even though the TestKit process might still be lingering.
• The jacoco plugin applied to the TestKit project ensures that the coverage is fully flushed after the test finishes. This ensures that the coverage is recorded even though the TestKit process might still be lingering.

See

• gradle/gradle#1465
• gradle/gradle#12535
• gradle/gradle#27328