Measure flakyness on Windows CI

[BraisGabin]

We all know that our Windows & JDK11 CI is flaky. It show flakyness. Here I have a list of 8 failed builds because of that:

• https://ge.detekt.dev/s/ptkpjsdzfqcuo/tests/overview
• https://ge.detekt.dev/s/nxk7tsjofd4sy/tests/overview
• https://ge.detekt.dev/s/aoeteqlgutbni/tests/overview
• https://ge.detekt.dev/s/ei3cidvaw6m3s/tests/overview
• https://ge.detekt.dev/s/635dpvmudbutq/tests/overview
• https://ge.detekt.dev/s/ymzazhyn74f3o/tests/overview
• https://ge.detekt.dev/s/6u2n3iuilq3lu/tests/overview
• https://ge.detekt.dev/s/77bto742izr4y/tests/overview

We had this problem already with Windows & JDK8 and we "fixed" it by disabling that combination: #4403

Right now we don't have flakyness information on of Gradle enterprise instance https://ge.detekt.dev/scans/tests because we need to activate this plugin: https://docs.gradle.com/enterprise/flaky-test-detection/

That's what this PR does, add that plugin. I don't like the idea of "retry" tests but I think that this is the only option that we have right now.

Open question: Should I enable it only for Windows?

[codecov]

Codecov Report

Merging #4742 (a98bb56) into main (a51adb8) will decrease coverage by 0.04%.
The diff coverage is n/a.

❗ Current head a98bb56 differs from pull request most recent head bde4400. Consider uploading reports for the commit bde4400 to get more accurate results

@@             Coverage Diff              @@
##               main    #4742      +/-   ##
============================================
- Coverage     84.74%   84.69%   -0.05%     
+ Complexity     3431     3418      -13     
============================================
  Files           491      490       -1     
  Lines         11270    11235      -35     
  Branches       2074     2068       -6     
============================================
- Hits           9551     9516      -35     
- Misses          673      675       +2     
+ Partials       1046     1044       -2     

Impacted Files	Coverage Δ
...gitlab/arturbosch/detekt/cli/ArgumentConverters.kt	69.23% <0.00%> (-4.11%)	⬇️
...les/style/ExplicitCollectionElementAccessMethod.kt	63.63% <0.00%> (-3.04%)	⬇️
...lab/arturbosch/detekt/rules/style/UnusedImports.kt	87.34% <0.00%> (-2.54%)	⬇️
...sch/detekt/rules/style/UnnecessaryAbstractClass.kt	79.16% <0.00%> (-2.47%)	⬇️
...ain/kotlin/io/gitlab/arturbosch/detekt/cli/Spec.kt	88.70% <0.00%> (-1.62%)	⬇️
...turbosch/detekt/rules/complexity/NamedArguments.kt	82.14% <0.00%> (-1.20%)	⬇️
...lab/arturbosch/detekt/rules/style/VarCouldBeVal.kt	80.41% <0.00%> (-1.15%)	⬇️
...n/io/github/detekt/report/html/HtmlOutputReport.kt	95.50% <0.00%> (-0.15%)	⬇️
...arturbosch/detekt/generator/printer/RulePrinter.kt	100.00% <0.00%> (ø)
...rturbosch/detekt/rules/style/StyleGuideProvider.kt	100.00% <0.00%> (ø)
... and 3 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 045928b...bde4400. Read the comment docs.

[runningcode]

👍

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Updated
detekt	✅ Ready (Inspect)	Visit Preview	Apr 25, 2022 at 7:29AM (UTC)

[chao2zhang]

Let's monitor Gradle enterprise / Test when this is merged. If this pipeline works, GE is an actual savior for monitoring and tackling test flakiness without manual CI scripting.