Static/Dynamic code analysis tools

[fredjeronimo]

In Prompt:

• Sonarqube - Explore

I've used quite a few other tools in the past such as Pc-Lint, NDepend, FxCop/StyleCop and Coverity. The latter is by far the most powerful one but it's quite expensive...

[smobs]

We use sonarqube on masketeers. Feels like RG has adopted this

[ChrisHurley]

I haven't had much value out of SonarQube but I think it's been tried fairly widely, yeah.

[nyctef]

+1 for not sure exactly how much value sonarqube provides - we have it on our PRs and it seems to occasionally pop up with some useful suggestions (and seems to be more intelligent / better at finding actual bugs than R# or compiler warnings) but we still spend a nonzero amount of time managing false positives and stuff we just don't care about.

I'd be interested in potentially exploring some of the alternatives @fredjeronimo mentioned to see if they do any better

[ChrisLambrou]

We use code inspections in R# and TeamCity in The Spiders. It's kind of annoying at times, but it's nonetheless keeping us honest for a good handful of basic code quality issues.

[garethbragg]

Considering how widely it's used, I do think Sonarqube should be in Adopt.

Quite happy for any alternatives to be in Explore.

[ChrisLambrou]

Ooops, I posted the following in #56, but it's probably more pertinent here:

@fffej, @garethbragg: FWIW, we've pretty much decided to ditch SonarQube analysis on SQL Data Catalog. It's just flaky as hell, causing many failed builds that are fixed simply by rerunning them, and it's never given us any value beyond the code inspections feature in R# and TeamCity, which is way more reliable.

Regarding the flakiness, look here: https://buildserver.red-gate.com/viewType.html?buildTypeId=CustomerDelight_SqlEstateManager_Web_2SonarqubeAnalysis
Every single failure comes with the error message "The SonarQube MSBuild integration failed: SonarQube was unable to collect the required information about your projects." I can't remember a time when it failed because of a legitimate code issue.

I'm a bit surprised this got promoted to Adopt to quickly given its luke-warm reception in this thread, and plenty of evidence that maintaining it is an annoyance and a distraction.

[nyctef]

Regarding the flakiness, look here: https://buildserver.red-gate.com/viewType.html?buildTypeId=CustomerDelight_SqlEstateManager_Web_2SonarqubeAnalysis

This appears to be the powershell + stderr + rg-buildphy02 problem that's been popping up recently: (eg [1] and [2]) - it's really annoying but possible to work around so that the build doesn't fail because of random warnings. Sonarqube is likely not doing anything wrong here.

I'm a bit surprised this got promoted to Adopt to quickly given its luke-warm reception in this thread, and plenty of evidence that maintaining it is an annoyance and a distraction.

Well, it got promoted to Adopt since it's currently widely adopted, and we're currently just trying to model the existing state of the world here. We can definitely look into exploring alternatives as well (I know several teams use R# inspections for a similar thing and I think people have looked into StyleCop/FxCop/etc).

Personally I think the sonarqube inspections are slightly better / more useful than other tools I've used, but the build issues are annoying, and it's only going to provide marginal benefit if you're using a similar tool already. Maybe it's just a case of picking one static analysis tool (per team) and sticking with it?

[ChrisLambrou]

If SonarQube requires a special workaround that no other tool seems to require, then maybe we should codify that workaround into a RedGate.Build cmdlet? e.g. invoke SonarQube analysis with -ErrorAction Continue, but separately capture any stderr output and subsequently raise an error directly in PowerShell if necessary? Would something like that solve the stability problems, or are they more wide reaching?

[ChrisLambrou]

Plus, adding support for SonarQube in RedGate.Build would mean I can get rid of stuff like this that I just don't want to have to maintain in our project's repo.

[nyctef]

If SonarQube requires a special workaround that no other tool seems to require, then maybe we should codify that workaround into a RedGate.Build cmdlet?

Git actually has the same problem with powershell and stderr as well (although it turns out that git has a workaround built-in through an environment variable you can set)

I agree we should probably pull something together in RedGate.Build for running sonarqube, though, for the reasons you said - it doesn't look like our various copies of sonarqube.tasks.ps1 have diverged much.

[ChrisLambrou]

I agree we should probably pull something together in RedGate.Build for running sonarqube, though, for the reasons you said - it doesn't look like our various copies of sonarqube.tasks.ps1 have diverged much.
👍

They haven't diverged "much", which means they might have a little bit. That's grounds enough to pull it into RedGate.Build. I think that's my 10% time job for tomorrow! 😀