Adds support for Bitbucket Cloud Code Insights

[marvin-w]

Intention

This PR intends to provide support for the code insights feature for bitbucket cloud.

History

Since about 2 months bitbucket cloud also has a code insights feature that one can use. After checking the differences between the cloud and the server implementation it is however not possible to completely reuse the server logic due to renamed/missing fields in the cloud version.

I haven't changed any public facing API (configs, properties, decorators) in this PR which allows @mc1arke to keep it up to date with the other sonarqube versions.

Additions

• The API is fully functional now for both the server and the cloud version. One development limitation is that Bitbucket actually checks the URL and doesn't allow localhost URLs. That's why someone testing this with a local instance or an instance whose DNS isn't publicly accessible (not confirmed) might have troubles.
• I tried to create a wiki page for this new decorator, however according to https://github.community/t/how-to-fork-a-wiki-and-send-a-pr/2006 it is currently not possible to fork a wiki repository. Is it possible to be granted access to feature branches for the wiki git - then I could create a PR. Alternatively, setting up a dedicated wiki repository could also make sense in case other people are interested in contributing to it.

Screenshots

[marvin-w]

@mc1arke I'm done with this. Feel free to review it once you have some time.

@singhsoldier13, @goober, @jburgel-davinci, @yeroc If there is a chance it would be awesome if you could also go for an additional test run with this version of bitbucket cloud PR decoration.

For the token please use base64(username:password) (At least Read repository scope is required)

For the URL please use https://api.bitbucket.org

[mc1arke]

I've given it a once-over review and it looks promising, but there are a few things that are likely to need changed. Please also have a look as Sonarcloud's review result and fix or respond to any comments it's raised. I'll give this a more thorough review once I've had a chance to run it.

[marvin-w]

@mc1arke I did a major refactoring considering your point about the race condition. Would you mind having a quick check again - I'd then finish all the tests and take care of sonar. I personally like it way more now.

[mc1arke]

Looks good from a quick read, although I've not run it yet so can't confirm it's functionally what I expect.

I've added a set of suggestions around how you've currently managed the BitbucketClient instance, sorry if I'm explaining an obvious concept, but didn't want to make a suggestion that wasn't understandable.

[marvin-w]

@mc1arke Thanks for the review again! I integrated your suggestions and fixed/adjusted the tests accordingly. Give it a go when you got some time :).

Please note that you need to set CloudCreateReportRequest.link to some valid link other than localhost for the API to fully function.

Otherwise you get below error:

sonarqube | com.github.mc1arke.sonarqube.plugin.ce.pullrequest.bitbucket.client.BitbucketCloudException: HTTP Status Code: 400; Message:{"key": "report-service.general.bad-request", "message": "link is not a valid URL", "arguments": {}}

I'll look at sonars report later since it's usually quite slow.

I also think its relatively easy to merge this into master afterwards as you could basically just replace the UnifyConfiguration with the AlmSettingDto.

[yeroc]

@marvin-w Would love to try this out. It looks like we'll need to be running SonarQube 7.8 or newer to try this out? We're running 7.7 right now so will need to upgrade first I guess?

[marvin-w]

Yes, you'll need to upgrade I believe. I didn't test it in lower versions.

[marvin-w]

@mc1arke would you mind retriggering the sonar analysis? For some reason it doesn't analyse the code in this branch.

[marvin-w]

@mc1arke I took care of the sonar stuff. I also tried to increase the coverage with my last commit.

I changed the underlying Mockito engine so that it now is possible to mock final classes (https://github.com/mockito/mockito/wiki/What%27s-new-in-Mockito-2#unmockable). Unfortunetly, this new engine broke some other tests which I fixed too. I deliberately put all that stuff in another commit as I wasn't sure whether you're up for that kind of change. Please restart sonarqube and then lets have a look at the coverage.

[mc1arke]

A few more comments - nothing too major; Your change generally looks good.

I'm not a fan of mocking final classes but I don't think you have much other option given the tight dependency on OkHttpClient. I'll need to look at how we can consistently use Http Clients in the plugin in a testable way, but that would be a separate change and not one that should impact your PR.

[marvin-w]

@mc1arke Thank you for your review. I took care of all the points you mentioned. Would you mind retriggering the sonar analysis? I added missing tests for the class BitbucketCloudException and the BitbucketClientFactory class.

Also, what is your opinion concerning the documentation mentioned in the description of this pull request? Since this feature is not officially supported by SonarQube (i.e. no official docs) it might become necessary.

I rebased all the commits into one and did a final test after the refactoring to be sure that the functionality is still working as intended.

[marvin-w]

@mc1arke You're probably a busy man and I'm sorry for bothering you again but do you have an ETA on when we can expect this to be merged and released?

[ajcamilo]

I've tested your PR with Sonarqube 7.9.3 with Bitbucket Cloud and it seems to work.

Except for the annotations. I don't know if it's something I need to change in bitbucket, but the annotations are not shown in the comments.

[marvin-w]

Cool, glad it works for you. There are no comments on files. The annotations belong to the report. In the report you can go directly to the specific file and line.

Please look at my screens in the PR description. If you still think there is an issue please add some more information to it, maybe a screenshot.

[ajcamilo]

You are right, sorry. I was looking at screenshots of the Bitbucket Server (adds annotations to the diff section)

Everything worked 👍

Maybe there should be some information for the format the bitbucket token (base64 user:pass), because this information will be lost in this PR.

[marvin-w]

Exactly that is what I meant in #185 (comment).

[sonarcloud]

Kudos, SonarCloud Quality Gate passed!

0 Bugs
0 Vulnerabilities (and 0 Security Hotspots to review)
0 Code Smells

83.6% Coverage
0.0% Duplication

[mc1arke]

Merged. Many thanks for the contribution!

Now that you're a contributor to this repository, I believe you should have access to edit the Wiki. If you don't then please let me know and I'll look at options that allow easier contribution to documentation.

[marvin-w]

@mc1arke Thanks for taking care of it. I have two more things to mention here.

1. I am able to pull the wiki but I can't edit it, neither in the UI nor in my local git clone. When forking the repo the wiki is also not included and I wouldn't be able to create a pull request. I could still pull your wiki and force push it to my fork, do the changes but you'll then need to do the merging manually. For me, personally, this would be a pain so I believe that there should be something else. I've used custom wikis deployed via https://www.netlify.com/ before with great success. If you're interested in that I could create you an initial PR and you could have a look. This would have the benefit that everyone can contribute to the documentation and you also have a live preview of all the changes in any given feature branch.

(Pushing to https://github.com/mc1arke/sonarqube-community-branch-plugin.wiki.git
remote: Permission to mc1arke/sonarqube-community-branch-plugin.wiki.git denied to marvin-w.
fatal: unable to access 'https://github.com/mc1arke/sonarqube-community-branch-plugin.wiki.git/': The requested URL returned error: 403)

2. Should I create a PR for the merge from release/1.3 to master in order to enable support for bitbucket cloud PR decoration also for Sonarqube >7.9 or do you want to take care of that yourself?

[yeroc]

@marvin-w @mc1arke Thanks so much for your work on this feature, I've built and tested this on a Sonar 7.9.3 (LTS) installation and it works great. Look forward to seeing this in a release to simplify deployment!

[anuragpathak21]

Hi @marvin-w / @mc1arke I am trying to use it for PR Decoration for gradle based Java Project. I have pulled the latest code build and deployed in the server and can it is properly showing the branches. I need help in setting it up for PR Decoration. The current Wiki does not have clear instruction to set this up. I will appreciate your help in this regard.

• Bitbucket Cloud (Private Repository)

• Sonarqube Server V 8.3.1

• Jenkins (Pipleine)

[marvin-w]

@anuragpathak21 This feature is currently only supported in the LTS 7.9.x version.

[anuragpathak21]

@anuragpathak21 This feature is currently only supported in the LTS 7.9.x version.

I am planning to downgrade to LTS 7.9. Will appreciate if you provide details of the setup.

[marvin-w]

For the general setup of the PR decoration please refer to the official SonarQube documentation.

For the specific feature you need a user with the "Read Repository" Scope. The token is defined as base64(username:password) of your technical user. I also wrote that #185 (comment) before (in this very thread).

For anything else please refer to the respective tool documentation, i.e. if there's something wrong in Jenkins ask them or if you also believe there should be PR refs in bitbucket cloud then this is the right way for you: https://jira.atlassian.com/browse/BCLOUD-5814 (open since 2012... you gotta love atlassian for that). Without this it is very hard to integrate the PR decoration with Jenkins as there are no dedicated git refs for pull requests.

[myDisconnect]

@yeroc @marvin-w how did you manage to build this under 7.9.3 (LTS)?
I tried changing the build.gradle -> sonarqubeVersion to 7.9.3 and got so many exceptions.

[yeroc]

@myDisconnect I suspect you're compiling the wrong branch. You need to clone/checkout and build the release/1_3 branch.

[marvin-w]

@mc1arke When can we expect this to be released? Is there anything we can help with?

[mc1arke]

@marvin-w This has now been released in 1.3.2 (Sonarqube 7.8-8.0) and 1.4.0 (Sonarqube 8.1). Sorry for the delay with getting the releases out, and thanks once again for the contribution.

On the documentation side: I'd like the documentation to be tightly linked to a release version of possible, so that it's clear how to configure a certain version of a plugin by looking at the relevant tag in Git. I'd therefore envisaged having a docs directory (or similar) in Git containing Markdown file. Does this cause any particular issues for you?

[marvin-w]

@mc1arke Thanks! No worries :)

I'll also have a look concerning the documentation then.

[rhaex]

@mc1arke I'm done with this. Feel free to review it once you have some time.

@singhsoldier13, @goober, @jburgel-davinci, @yeroc If there is a chance it would be awesome if you could also go for an additional test run with this version of bitbucket cloud PR decoration.

For the token please use base64(username:password) (At least Read repository scope is required)

For the URL please use https://api.bitbucket.org

Could this information please be added to the readme.MD file? As it points to the official documentation, but the official version does not support BitBucket cloud, so the only place where to find these parameters right now is this pull-request.

Also, does the base64(username:password) need to be added literally in that field, or do you mean you need to manually base64 the values for username:password an place that in the field?

[nfalco79]

@marvin-w if this PR also approve the Pull request (configurable) without any kind of comment was great. Because Repose code insight does not play in any way in branch restriction.

[marvin-w]

@marvin-w if this PR also approve the Pull request (configurable) without any kind of comment was great. Because Repose code insight does not play in any way in branch restriction.

I read your post three times and I don't get what you are trying to tell your audience in a PR that has been closed for over a year.

[nfalco79]

I would say that approval should not be discarded as idea. Ok that bitbuket "Code insights" is the tool that bitbuket provides to report any kind of metric but it does not play any role in branch restrictions. This means for example that you could have a PR green just because the code compile, regardless if Sonarqube reports failures or not. PR could be merged and there are not option to prevent this. The approval instead could play a role in a branch restrictions. I mean the approval code could be left there (configurable) based on quality gate.

[marvin-w]

Please open an issue for additional feature requests or bug reports.

[nfalco79]

We have migrated the sonarqube version for a month from the mibexsoftware plugin to this one and I must say that most developers are still puzzled to understand when there are actually analisy sonars. Both because the Pull Request Decoration must be configured in each project (and this is a problem because it must be done after the project has been created) to see the reports and because it is not very visible in BB Cloud.
We then had to implement the sonar analysis pipeline stage decoration as failed (but not the build) and then implement the PR approval by waiting for the analysis id to be available to reach the minimum 2 approvals for a PR, one automatic (build system) and the other of another real person. If it had been possible to enable approval directly in the plugin it would have been easier since the plugin already has all the necessary information.

Please open an issue for additional feature requests or bug reports.

Sure I will do

[nfalco79]

Issue #77 is still open and summarize above requirement