Allure gradle plugin causes execution of additonal test sets as part of build task

[benjamindibonasonos]

I'm submitting a ...

• bug report
• feature request
• support request => Please do not submit support request here, see note at the top of this template.

What is the current behavior?

After upgrading Allure gradle plugin from 2.8.1 to 2.9.4, the gradle build task now includes compilation and execution of test sets other than unit tests.

We use the gradle test sets plugin to manage our integration tests, which states

By default, the tests in a custom test set are not executed when you call gradle build.

https://github.com/unbroken-dome/gradle-testsets-plugin

Reverting back to allure-gradle 2.8.1, the build task only executes the unit tests, which is the expected behavior.

If the current behavior is a bug, please provide the steps to reproduce and if possible a minimal demo of the problem

• Add integration test set to build.gradle - https://github.com/unbroken-dome/gradle-testsets-plugin
• Run ./gradlew clean build
• Observe execution of integration tests

What is the expected behavior?

Gradle build task should only execute unit tests.

Please tell us about your environment:

| allure-gradle version | 2.9.4 |
| Gradle version | 6.9 |
| Unbroken Dome Test-Sets version| 4.0.0 |
| Test framework | JUnit 5 |

Other information

We have included a testSets configuration in build.gradle like the following:
testSets { functionalTests {} }

[NielsDoucet]

We have noticed the same behaviour, where tests are being run when requesting the assemble task.
After discussing with @vlsi on the gradle community slack, it appears this is related to gradle/gradle#6875, as allure results are attached to the build as an artifact.

@benjamindibonasonos Could you verify that you also see those tests executed if you run ./gradlew assemble?

[vlsi]

If you need to configure allure for certain tasks only, I would suggest the following:

allure {
    adapter {
        // Disable automatic configuration
        autoconfigure.set(false)

        // Gather report data from test task only
        gatherResultsFrom(tasks.test)
        addAspectjTo(tasks.test)
    }
}

However, it might be worth adding a slightly better DSL instead of gatherResultsFrom and addAspectjTo .

[benjamindibonasonos]

We have noticed the same behaviour, where tests are being run when requesting the assemble task. After discussing with @vlsi on the gradle community slack, it appears this is related to gradle/gradle#6875, as allure results are attached to the build as an artifact.

@benjamindibonasonos Could you verify that you also see those tests executed if you run ./gradlew assemble?

@NielsDoucet I can confirm that these tests are executed with the "assemble" task.

./gradlew assemble --console=plain                                                                                              
> Task :bootBuildInfo
> Task :compileJava UP-TO-DATE
> Task :processResources
> Task :classes
> Task :bootJarMainClassName
> Task :bootJar
> Task :compileFunctionalTestJava
> Task :processFunctionalTestResources UP-TO-DATE
> Task :functionalTestClasses
> Task :functionalTest