Support configurable severity per ruleset/rule in XML and Sarif output

[picklebento]

Summary

This PR implements configuring the severity in the XML and Sarif output, which is the first step to implement #3274. The implementation is also different from #3306, because the severity in the configuration can only be a simplified version and does not allow custom strings.

Implementation

Adding the severity as a property of Issue does not work: Each rule initializes the issue first then uses the issue.id to initialize ruleConfig. If we want to use ruleConfig to parse severity, it has to be done after issue is created.
We can break such cycling dependency by refactoring, but I would prefer doing so in the next PR where we can remove the existing Severity and replace it with the new SeverityLevel.

The choice here is to add the severity as a property of Finding, which is appended at the end to mitigate incompatibility. However, since Finding and its children are not data class, I need to create copyWithSeverity to override the given severity.

Followup

I did not deprecate or remove Severity, maxIssues, warningsAsErrors in this PR. If this is a good time to do so, I would be happy to do those in a follow-up PR.

Test

In addition to the unit test, I also performed an integration test in a different repo. Users can configure severity at ruleset level and rule level.

style:
  severity: error
  MagicNumber:
    severity: ignore

[codecov]

Codecov Report

Merging #3310 (afcab04) into master (ff655d2) will decrease coverage by 0.07%.
The diff coverage is 70.37%.

@@             Coverage Diff              @@
##             master    #3310      +/-   ##
============================================
- Coverage     80.34%   80.27%   -0.08%     
+ Complexity     2724     2723       -1     
============================================
  Files           445      445              
  Lines          8177     8193      +16     
  Branches       1555     1558       +3     
============================================
+ Hits           6570     6577       +7     
- Misses          774      778       +4     
- Partials        833      838       +5     

Impacted Files	Coverage Δ	Complexity Δ
...n/kotlin/io/gitlab/arturbosch/detekt/api/Config.kt	100.00% <ø> (ø)	0.00 <0.00> (ø)
...lin/io/gitlab/arturbosch/detekt/api/ConfigAware.kt	66.66% <ø> (ø)	0.00 <0.00> (ø)
...kotlin/io/gitlab/arturbosch/detekt/api/Findings.kt	62.50% <0.00%> (-8.93%)	0.00 <0.00> (ø)
...lin/io/github/detekt/report/xml/XmlOutputReport.kt	90.90% <0.00%> (-5.25%)	6.00 <1.00> (-3.00)
...otlin/io/gitlab/arturbosch/detekt/api/CodeSmell.kt	59.57% <50.00%> (-0.90%)	13.00 <2.00> (+2.00)	⬇️
...ain/kotlin/io/gitlab/arturbosch/detekt/api/Rule.kt	83.33% <60.00%> (-16.67%)	16.00 <3.00> (+1.00)	⬇️
...sch/detekt/core/config/ValidatableConfiguration.kt	96.96% <100.00%> (+0.09%)	0.00 <0.00> (ø)
...io/github/detekt/report/sarif/SarifOutputReport.kt	100.00% <100.00%> (ø)	4.00 <0.00> (ø)
...in/kotlin/io/gitlab/arturbosch/detekt/api/Issue.kt	75.00% <0.00%> (-12.50%)	4.00% <0.00%> (-1.00%)

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ff655d2...afcab04. Read the comment docs.

[picklebento]

I also realized that the severity, for now, is only in XML + Sarif. We are not differentiating severities in HTML or console output. If this is a good idea, please react to this comment so I will open a new issue for supporting them.

1. For txt, the output should include the severity level.
2. For console, we should color-code each finding in addition to showing the severity.
3. For HTML, we should both display the severity and use a different background.

[BraisGabin]

I agree with all except html. I don't think that we should change all the background. We could use a label and only change that label color.

[schalkms]

Looks good!
Anyhow, a refactoring would be great in separate PR in order to implement the severity levels properly and keep the architecture tidy. Then there's no need for copying stuff around and no risk to introduce cyclic dependencies with future refactorings.

We can break such cycling dependency by refactoring, but I would prefer doing so in the next PR where we can remove the existing Severity and replace it with the new SeverityLevel.

[schalkms]

If this is a good idea, please react to this comment so I will open a new issue for supporting them.

1. For txt, the output should include the severity level.
2. For console, we should color-code each finding in addition to showing the severity.
3. For HTML, we should both display the severity and use a different background.

+1 great
I agree with this suggestion. I wouldn't want to build it overly complicated, so simplicity is the key here.

[picklebento]

Looks good!
Anyhow, a refactoring would be great in separate PR in order to implement the severity levels properly and keep the architecture tidy. Then there's no need for copying stuff around and no risk to introduce cyclic dependencies with future refactorings.

We can break such cycling dependency by refactoring, but I would prefer doing so in the next PR where we can remove the existing Severity and replace it with the new SeverityLevel.

#3127 prototyped a new Rule API, which is much cleaner, but also not backward compatible. In order to keep the severity within every Issue, it will be a breaking change and I would prefer to wait till Detekt 2.0 for making such changes.

I have seen Detekt 2.0 mentioned in a few issues. Do we know when and how we are planning this? I imagine there will be a separate 2.0 branch where we can merge all breaking changes.

[schalkms]

#3127 prototyped a new Rule API, which is much cleaner, but also not backward compatible. In order to keep the severity within every Issue, it will be a breaking change and I would prefer to wait till Detekt 2.0 for making such changes.

I have seen Detekt 2.0 mentioned in a few issues. Do we know when and how we are planning this? I imagine there will be a separate 2.0 branch where we can merge all breaking changes.

At the moment those PRs are tagged in the milestone field with the corresponding detekt version.
Nevertheless, with the number of those PRs growing I suggest to create a new v2.0 branch for simplicity reasons.

[BraisGabin]

We talk a lot about 2.0 but we don't have the list of things that we want to change. So I'm not sure about creating a new branch. #3127 pretends to be retrocompatible. I mean, the idea is to deprecate the old api and keep it until 2.0.

[arturbosch]

Implementation looks good!
I think this could be backwards compatible for 1.x, please see my two comments below.

I also added a comment to our 2.0 ticket - #2680 (comment)

[BraisGabin]

Great PR! I'm just a bit concerned about the compatibility.

[picklebento]

This PR also depends on #3320