SonarQube support

[shadow7412]

Context

Although I've successfully managed to get sonarqube analysing in my gitlab pipeline, the runs could be made significantly faster by adding some commands to the docker image.

Suggested solution

If the following two steps current implemented in my pipeline were baked into the image instead, each build would take about 3 minutes less.

• apt-get install -y openjdk-11-jre
• dotnet tool install --global dotnet-sonarscanner

Even just priming dotnet (as it takes an extra minute to run the first time you use it even if you don't do anything meaningful) would be a big help.

Considered alternatives

• I could make an image that uses gameci as a base, but I wanted to put this out there because I think quite a few people may benefit from these changes.
• Instead of bloating the existing images, we could add another "target platform" for sonar builds. This would probably be the best solution, as we wouldn't bloat existing images but still would give the desired build time benefits to those who desire them.

Additional details

It could be worth a conversation about adding an official guide to using sonarqube to the main documentation. That would encourage people to use it (and thus write better code) and make it easier for people like me who had to fight the dragons in near solitude.

I'm happy to do the PR for this one - but I wanted some feedback first before putting in the effort.

[shadow7412]

Update: I've gone with alternate number 1 as a stop-gap. You can view it here if interested.

There's probably more we can do to help one who uses this image (such as set DOTNET_ROOT), but this alone wipes quite a measurable amount of time off an individual build.

[webbertakken]

I probably should have posted this comment here. #74 (comment)

[webbertakken]

This comment should probably also have been here. #74 (comment)

I did not realise building the project is a requirement for a static code analysis tool like Sonarqube. It feels a bit off to me. Are there docs we can refer to that explain this?

[shadow7412]

You can find a more generic example of a C# project on their website.

But the basic recipe is:

• Run the sonarqubestart command. This read the configuration from the sonarqube server, and makes changes to your build config so that more information is captured, and put into specific files for later analysis.
• Run the normal build command (which has much more output than it normally would)
• Run the sonarqubestop command. This collates all the output, creates reports, then uploads it to the sonarqube server. It also reverts any changes it made to the build config.

In case it helps explain it further, the actual commands I'm using in my pipeline are as follows;

First, I'm using my custom image which basically runs the following two commands. You'd need to do this manually if running on the normal editor image.

apt-get update && apt-get install -y openjdk-11-jre
/opt/unity/Editor/Data/NetCore/Sdk-*/dotnet tool install --global dotnet-sonarscanner

Then, my actual pipeline.

# Run `UnityEditor.SyncVS.SyncSolution` so that the sln and proj files are generated.
- chmod +x ./ci/generate_sln.sh && ./ci/generate_sln.sh

# Add dotnet to the path, and set the DOTNET_ROOT as it's in a nonstandard directory.
- "export PATH=\"$PATH:`echo /opt/unity/Editor/Data/NetCore/Sdk-*/`:$HOME/.dotnet/tools\""
- "export DOTNET_ROOT=\"$(dirname $(which dotnet))\""

# dotnet freaks out if there are multiple csprojs in the working directory.
# so to avoid this issue, I just create a temporary directory to step into.
# As I'm specifying the sln, it doesn't matter to the build.
- mkdir sonar && cd sonar

# Finally, run sonarqube.
- "dotnet sonarscanner begin /k:\"$SONAR_PROJECT_KEY\" /d:sonar.login=\"$SONAR_TOKEN\" /d:\"sonar.host.url=$SONAR_HOST_URL\""
- "dotnet build ../*.sln"
- "dotnet sonarscanner end /d:sonar.login=\"$SONAR_TOKEN\""

[shadow7412]

Option 1 presented above is a terrible idea. It turns out that the act of installing java alone boosts the image size by a GB. Stupid java...

EDIT: I read the wrong number. Which honestly makes a lot sense - 1gb seems pretty heavy, even for java. But in total, these commands still add about 350MB. Therefore I think the concern still holds.

So I'm now advocating alternative option 2 - a "target" of sonarqube. This could be implemented in the way I've done it here (plus the additional actions etc to multiple that out by image and unity version), and we'd probably also want to make the supporting scripts available in the documentation somewhere.

[davidmfinol]

I like this second option of creating a SonarQube "target".
I recently got SonarQube without using docker images, but it would be nice to have images for SonarQube, to avoid having to download and install the missing the tools during the CI run.

[shadow7412]

For what it's worth, my implementation of this broke soon after unity 2020.1.13f1 cause they removed stuff from their install I was relying on (from memory, I think it was NetCore).

For now I've basically given up on it because of unity being a moving target, and the seeming lack of interest in this ticket. But I do miss having this working, so I'll still keep an ear out if people have any suggestions on how we can get this working in a more permanent way. I'm going to leave it open for the same reason - this would be a valuable addition to the game-ci suite.

[IARI]

I have made a sonarcube image based on the editor-images, running fine for me with 2020.3
Still working on getting resharper to run with it whenever i have time for that...
if anyone's interested I can post my docker files or upload something to docker hub.

[davidmfinol]

I'd be interested!

[shadow7412]

I have made a sonarcube image based on the editor images

Would you mind checking out the referenced a few comments ago, and consider raising a PR?

[IARI]

I have posted my docker files in a gist here:
https://gist.github.com/IARI/e588e07c75d267bfd6e171f19e5e585b

@shadow7412 I would doubt that this is fit for a PR in its current state - tbh I have not checked out the unityci dockerfiles so far - but at the very least, there must a way to deal with the combinatorial plethora of versions of the components involved. images would probably have to be generated for some if not many of them.

Additionally I highly anticipate Unity's move towards CoreCLR for .NET 6/7 - this will make running sonarscanner and resharper a bit easier.
https://forum.unity.com/threads/unity-future-net-development-status.1092205
waiting for the video version of Josh Petersons talk from yesterday at GDC:
https://schedule.gdconf.com/session/the-future-of-net-and-unity-presented-by-unity/886907?_mc=sem_gdcsf_sem_x_le_tspr_brndgl__2022

[shadow7412]

I hope you're right. Also it'd be nice to be finally be able to use the new C# language features that my IDE keeps dangling in front of my face, but unity does not support.

[shadow7412]

On the subject of possible alternatives, it looks like Jetbrains have started a project called Qodana which looks to have many of the same values as sonarqube. But their C# support could be a while away yet. But given their investment into unity in the form of rider and riderflow, it wouldn't surprise me to see unity being a first-class citizen there at some point.

That's probably going to be a far superior option in the long run.

[IARI]

Oh wow, I missed that! While I love jetbrains' Rider and all their other products that I use, I want to avoid complete vendor lock-in.
That being said, another option sounds like awesome news.

Anyways, in the meantime we continue with sonarqube.
Right now we are still not really utilizing its capacity at this point - also I haven't gotten Resharper to work properly with sonarqube yet, the whole codebase is full of old code and we need to agree on what rules we do want.
On that subject, I would really love to have some exchange. If any of you guys are interested, you can hit me up on unitydevs.slack.com @IARI (UA56HF4PK)

[webbertakken]

I've added this to the Roadmap for v3.0.0

It will be done as part of game-ci/cli#11

[shadow7412]

@webbertakken fantastic. It's mentioned above, but I'll reiterate that there is a working version of this over here that you should be able to draw inspiration from. I'll be more than happy to help out if required.

[webbertakken]

We are looking to write a plugin system directly into unity-builder that would install this during runtime in v3.0.0.

Other than that we have an open issue in documentation about customising images that can still be improved. Especially with this point we could definitely use your help.