Setup Gradle functional tests

[3flex]

This is a step on the way to separating Gradle TestKit tests from the other Gradle plugin tests (see #3778).

This PR sets up the Gradle build to support the new structure, and moves tests that use TestKit but don't use dry-run mode into src/functionalTest. It also moves shared code into src/testFixtures so they can be used by both functional tests and standard tests while that's still necessary.

There's a new task functionalTest which runs all tests in src/functionalTest. Both functionalTest and test are dependencies of check task and therefore also the build task.

[codecov]

Codecov Report

Merging #4074 (c387461) into main (231ba0f) will increase coverage by 84.27%.
The diff coverage is n/a.

@@             Coverage Diff             @@
##             main    #4074       +/-   ##
===========================================
+ Coverage        0   84.27%   +84.27%     
- Complexity      0     3249     +3249     
===========================================
  Files           0      471      +471     
  Lines           0    10261    +10261     
  Branches        0     1805     +1805     
===========================================
+ Hits            0     8647     +8647     
- Misses          0      664      +664     
- Partials        0      950      +950     

Impacted Files	Coverage Δ
...sch/detekt/rules/style/UnnecessaryAbstractClass.kt	84.78% <0.00%> (ø)
...osch/detekt/rules/style/OptionalAbstractKeyword.kt	89.47% <0.00%> (ø)
...lin/io/gitlab/arturbosch/detekt/rules/TypeUtils.kt	100.00% <0.00%> (ø)
...turbosch/detekt/rules/style/UseIfEmptyOrIfBlank.kt	80.00% <0.00%> (ø)
...bosch/detekt/formatting/wrappers/ImportOrdering.kt	100.00% <0.00%> (ø)
...b/detekt/metrics/processors/ProjectLOCProcessor.kt	100.00% <0.00%> (ø)
...in/kotlin/io/gitlab/arturbosch/detekt/core/Junk.kt	100.00% <0.00%> (ø)
...lab/arturbosch/detekt/rules/style/VarCouldBeVal.kt	89.58% <0.00%> (ø)
...n/kotlin/io/github/detekt/tooling/api/DetektCli.kt	100.00% <0.00%> (ø)
...b/detekt/tooling/internal/DefaultAnalysisResult.kt	87.50% <0.00%> (ø)
... and 461 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 231ba0f...c387461. Read the comment docs.

[chao2zhang]

IntelliJ IDE does not really like addition test source set like intTest. I just applied this PR locally and the problem mentioned in #3504 still persists.

I am thinking if we should start experimenting with extracting integration tests into a separate module as discussed in #3324 (comment).

[3flex]

I just applied this PR locally and the problem mentioned in #3504 still persists.

I'm not seeing that - which IDE specifically (IntelliJ, Android Studio) & which version? Can you try deleting the .idea directory and clean IDE cache?

I have no red underline in any files under /src/intTest (IntelliJ IDEA 2021.2.1) with almost all settings at defaults:

[chao2zhang]

After ./gradlew clean and removing .idea, the IDE does not seem to intTest as a test module (With green background).

Previously IntelliJ cannot infer the module dependencies correctly, and now after regenerating the .idea folder, it seems to work for me for the most part.

[3flex]

I think as long as the Spek IDE plugin can pick up the tests, Gradle executes them and IDE indexing works, the fact that it's not specifically identified as green/test module isn't a showstopper (it does bug me as well though! But this setup is simpler to me than having a whole new module for integration tests)

We can have others weigh in of course.

[cortinico]

the fact that it's not specifically identified as green/test module isn't a showstopper

Agree. Also I believe you can workaround the coloring issue with something like:

idea.module {
    sourceDirs.remove(file("src/intTest/kotlin"))
    testSourceDirs.add(file("src/intTest/kotlin"))
}

Sadly I wasn't able to let it work (see https://youtrack.jetbrains.com/issue/IDEA-151925). If you folks look into it let me know 🙏

[3flex]

Converting back to draft and removing from 1.19.0 milestone (for now) - Gradle 7.3 is introducing support for test suites which I believe supersedes the currently recommended configuration I've used here.

It would be better to wait to see what that looks like and use the newer, hopefully simpler setup.

[chao2zhang]

@3flex Would you mind sharing some of the pointers how Gradle 7.3 changes the test suites? (Assuming this is for Gradle integration test only)

[3flex]

Would you mind sharing some of the pointers how Gradle 7.3 changes the test suites?

https://docs.gradle.org/7.3-rc-1/release-notes.html#test-suites

[cortinico]

Great stuff! What's the status on the IDE support?

[3flex]

What's the status on the IDE support?

Unfortunately it's not configured automatically but I've requested this integration & configuration be applied automatically: gradle/gradle#18723