Should we keep SonarCloud?

[Richard-Gist]

Unlike last time when we asked this question to see if we should move to a different solution, I am proposing a different question here. The question is should we keep SonarCloud at all? As in, could we just remove SonarCloud without replacing it.

Tangentially this discussion may spread out into static analysis as a whole, and that's ok. But I think this question will be resolved when we know what we want to do about SonarCloud and then also, issue #20.

[Tyler-Keith-Thompson]

The things SonarCloud provides that nobody else does the same way, at all, or as well:

• More comprehensive static analysis rules
• The ability to leave comments about WHY we chose to ignore a static analysis issue
• Some primitive security checking (e.g. "Uh oh, I saw regex, regex can be bad...did you do a bad thing or is this regex okay?)
• The ability to gather metrics on NEW CODE and on all code.
• The ability to create custom metrics that'll fail a "quality gate". Like coverage on new code has to be > some %, duplications have to be lower than some %.

Workarounds, if you don't have SonarCloud anymore.

• SwiftLint gives you a bunch of rules already, you'll lose some of what SonarCloud has, but we already have linting as part of the pipeline and it already fails if any rule is violated (even warnings).
• You can leave comments about why we chose to ignore a SwiftLint rule in code, so that's nice.
• The security checking on this project probably isn't super worthwhile because PRs bring visibility to code before its merged in. All SonarCloud could ever really do is say "Look at this thing and decide if it's okay" which is precisely what a PR is all about.
• If you care about gathering metrics on new code I've gotta say PRs are your thing again. You know exactly what the new code is by looking at the diff between branches. It may be somewhat of a burden to compare on your own, but it's certainly possible.
• There are tools that'll fail the pipeline if coverage is too low already. Like this one or this one.
• I have never used duplication detection because it hasn't been a problem for me before, but here's an article and a tool

NOTE: Other things like cyclomatic complexity are actually covered by SwiftLint to some extent. SonarCloud does have a really unique metric that I like called "Cognitive Complexity". That being said it's all just a way of saying "You've got too many indents/branches for this to make sense".

All this to say I think you can manage just fine without SonarCloud. There are other options, some of them may be worth it and some really may not be.

[morganzellers]

I'll +1 this sentiment - I think we can get enough of the same good SonarCloud functionality into our pipeline without actually using SonarCloud.

[wiemerm]

That answered the questions I had really well. Looking into coverage options would be good as those seem to be predominately what we'd be losing that isn't already covered by either our current tools and/or processes. I agree that we could move away from SonarCloud.

[Richard-Gist]

I think CodeCov.io would work for what I would like as a minimum to replace SonarCloud, specifically the coverage of our tests.

+1 to removing SonarCloud.

[brianlombardo]

+1 for removing SonarCloud

[Richard-Gist]

Since we're now talking about switching to CodeCov, here is a PR with a sample report: cucumberswift/CucumberSwift#30

[Richard-Gist]

The answer to this as discussed by the team was to Remove SonarCloud and replace with Codecov.