Not failing or not using baseline

[fluxxion82]

Expected Behavior

detekt should run using the baseline and fail when issues are found if fastFail is set to true.

Observed Behavior

If I don't set fastFail, then detekt runs and will not fail when issues are found. If I set fastFail = true, then detekt doesn't seem to use the baseline file, or somehow find new issues

Steps to Reproduce

In my buildSrc module, I include 'io.gitlab.arturbosch.detekt:detekt-gradle-plugin:1.15.0' in the dependencies. Then in a custom plugin class that all my project modules use, I have

pluginManager.apply(DetektPlugin::class.java)
tasks.withType(Detekt::class.java).configureEach {
    it.jvmTarget = "1.8"
    it.config.from(rootProject.file("detekt-config.yml"))
    it.reports.apply {
        html.enabled = false
        xml.enabled = false
    }
    val file = File(rootProject.projectDir, "detekt-baseline.xml")
    it.baseline.set(file)
}

Next, I run detekt.

Context

I'm trying to include the detekt task in my CI for pull request, etc, but if the task doesn't fail, we won't know there are issues.

Your Environment

• Version of detekt used: 1.15.0
• Version of Gradle used (if applicable): 6.5
• Version of Gradle plugin: 4.0.1
• Operating System and version: Mac OS, catalina
• Link to your project (if it's a public repository):

[chao2zhang]

Thank you for submitting the bug report. I was not able to reproduce: With baseline set, I tried both fastFail = true and not setting fastFail, both yields new findings on top of the baseline.

If possible, would you mind creating a sample project to reproduce?

[fluxxion82]

Yeah, I'm attaching a basic project that has a similar structure to my current project. I've been playing with detekt versions 1.14.0 and 1.15.0, and I've been testing the failFast config set to true and false, but each set up passes, except with 1.15.0 and failFast = true, then I get some weird exception that doesn't seem related to the format checking, or at least it doesn't show the issues and says that it failed while analyzing a particular file that doesn't have any issues. I thought I had avoided this exception at some point but not sure how I did it. There are some weird exceptions that get printed out with failFast = true on 1.14.0, but the task still succeeds.

The main place you want to look for testing is in the buildSrc module. You can change the version of detekt in the build.gradle file, and in the class BasePlugin, take a look at the function configureDetekt(). I've tried setting the detekt plugin in the build script and using the detekt extension in the plugin class. I think when I used the plugin/extension, the baseline wouldn't get used when failFast is set to true, but I think uses it when it's false. And again, in either case the task doesn't fail.

I put the sample project on my google drive. Let me know if you have issues getting it or have questions about the setup.
https://drive.google.com/file/d/1ZGxgRzUi15imIIWM7mJBm1XSsp73NTOF/view?usp=sharing

Thanks

[fluxxion82]

Hello, I was wondering if you had a change to look at the sample project, plus if you were able to reproduce the issue with it and had any ideas yet about what might be going on. Thanks.

[chao2zhang]

Sorry this falls through the cracks, I have requested access to the Google drive.

[chao2zhang]

When on 1.15.0 and failFast = true, the exception below tells us that the yaml config has undesired types:

> Analyzing /Users/cazhang/Downloads/DetektTest/app/src/main/kotlin/com/example/detekttest/MainActivity.kt led to an exception. 
  The original exception message was: Value "[**/test/**, **/androidTest/**, **/commonTest/**, **/jvmTest/**, **/jsTest/**, **/iosTest/**]" set for config parameter "comments > excludes" is not of required type String.
  Running detekt '1.15.0' on Java '1.8.0_212-b10' on OS 'Mac OS X'
  If the exception message does not help, please feel free to create an issue on our GitHub page.

comment:
  // OK
  excludes: [".*/test/.*", ".*Test.kt", ".*Spec.kt"]
  // Fail
  excludes: ".*/test/.*, .*Test.kt, .*Spec.kt"

In addition, failFast is going to be deprecated as the naming was confusing. See #3391

[chao2zhang]

When on 1.15.0 and failFast = false (This is the recommended setting), it seems like ./gradlew detekt or ./gradlew projectDetekt does respect baseline. The task succeeds because detekt-config.yml has build.maxIssues: 1 and there is exactly one issue found.

I would change the maxIssues to 0. Let me know if this doesn't help.

[fluxxion82]

Awesome, all of this has fixed the issue. Thanks a bunch for taking a look at this.