Jacoco Gradle Sonarqube multi-module

[marcofranssen]

I'm unable to get a working code coverage report for sonarqube plugin so all the covered code is reflected. There are always some of my subprojects missing from the coverage report which is invalid as they are touched by my tests for sure.

Is there anyone able to provide a good example on the official documentation page over here?

https://docs.gradle.org/current/userguide/jacoco_plugin.html

It seems there are many people suffering from this, all providing different approaches, all giving different, but inaccurate results. With one approach Project B isn't in the coverage report with another approach it appears Project C isn't included etc.

Would be great if there is a complete example on how to use jacocoMerge tasks with gradle 5.3 using a gradle project with subprojects.

Below an extract of my build gradle with the last approach I tried for getting the gradle stuff merged.

plugins {
    id "org.sonarqube" version "2.7"
    id 'jacoco'
}

def allTestCoverageFile = "${buildDir}/jacoco/allTestCoverage.exec"

task jacocoMerge(type: JacocoMerge, group: 'verification') {
    subprojects.each { subproject ->
        executionData subproject.tasks.withType(Test)
    }
}

sonarqube {
    properties {
        property "sonar.scm.provider", "git"
        property "sonar.jacoco.reportPaths", allTestCoverageFile
    }
}

subprojects {
    apply plugin: 'java'
    apply plugin: 'jacoco'

    sonarqube {
        properties {
            property "sonar.jacoco.reportPaths", allTestCoverageFile
        }
    }
}

project(':projectA') {
    dependencies {
        compile project(':projectB')
        compile project(':projectC')
    }
}

project(':projectB') {
    dependencies {
        compile project(':projectC')
    }
}

project(':projectC') {
    dependencies {
    }
}

[marcofranssen]

After a 2 days of trying different things I found the following to be working. Not sure if this is the best approach, but this at least works for me now.

I think the key is to have the jacocomerge being called using the test.finalizedBy(jacocoMerge) to get it working. I also changed the executionData line in jacocoMerge based on another example I found from someone else.

plugins {
    id "org.sonarqube" version "2.7"
    id 'jacoco'
}

repositories {
    mavenCentral()
}

def allTestCoverageFile = "${rootProject.buildDir}/jacoco/allTestCoverage.exec"
task jacocoMerge(type: JacocoMerge, group: 'verification') {
    destinationFile = file(allTestCoverageFile)
    executionData = project.fileTree(dir: '.', include:'**/build/jacoco/test.exec')
}

jacoco {
    toolVersion = "0.8.3"
}

sonarqube {
    properties {
        property "sonar.scm.provider", "git"
        property "sonar.jacoco.reportPaths", allTestCoverageFile
    }
}

subprojects {
    apply plugin: 'java'
    apply plugin: 'jacoco'

    sonarqube {
        properties {
            property "sonar.jacoco.reportPaths", allTestCoverageFile
        }
    }

    test.finalizedBy(jacocoMerge)
}

project(':projectA') {
    dependencies {
        compile project(':projectB')
        compile project(':projectC')
    }
}

project(':projectB') {
    dependencies {
        compile project(':projectC')
    }
}

project(':projectC') {
    dependencies {
    }
}

Is this the best approach to do this? If yes it might make sense to add this to the jacoco plugin documentation as an example to use it for multimodule gradle projects.

[TimvdLippe]

I am facing similar difficulties (mockito/mockito#1689). I will try out your approach and report if it is working.

[zgranik]

It works for me also

[hajush]

After upgrading to SonarQube 8.1 (which includes some show unstopping bug fixes, so downgrading isn't an option), the binary jacoco exec data is no longer supported, which means the JacocoMerge solution from the jacoco plugin no longer works. And we're losing our coverage tracking for our large multi module project since the upgrade.

[Veranicus]

Can I please find somewhere basic process description on how to implement jacoco and sonar for multi module gradle project ? I have found so many misleading and contradictory information...

[marcofranssen]

@Veranicus at least my comment above did work for us at the time of writing. Not sure with newer versions etc. Can't check either as I left the company and thereby also that codebase.

#8881 (comment)

[hajush]

@marcofranssen your solution is appropriate only for older versions of the SonarJava scanner. The latest does not support the old jacoco binary .exec format. It requires XML. The solution provided at https://community.sonarsource.com/t/coverage-test-data-importing-jacoco-coverage-report-in-xml-format/12151 describes how to support multi-module xml merge with Maven, but it is not explicit with Gradle. Removing the JacocoMerge step and ensuring the xml data is being produced, we are getting coverage numbers again. But nearly 1/2 of what they used to be.

[ParanoidUser]

@hajush do you see any reason why the coverage values have changed?

[hajush]

I’ve not worked it out yet but has everything only done limited experiments. I suspect it is just a path issue. I plan to spend more time investigating next week and will update what I find.

[ashwini-desai]

I've got it working finally after a lot of struggle and referring to documents and solutions.

My build.gradle(root build.gradle):

subprojects {
    apply(plugin: 'org.jetbrains.kotlin.jvm')

    repositories {
        jcenter()
        mavenCentral()
   }
}

task codeCoverageReport(type: JacocoReport) {

    // Gather execution data from all subprojects
    executionData fileTree(project.rootDir.absolutePath).include("**/build/jacoco/*.exec")

    // Add all relevant sourcesets from the subprojects
    subprojects.each {
        sourceSets it.sourceSets.main
    }

    reports {
        xml.enabled true
        html.enabled true
        csv.enabled false
    }
}

// always run the tests before generating the report
codeCoverageReport.dependsOn {
    subprojects*.test
}

sonarqube {
    properties {
        property "sonar.projectKey", "your_project_key"
        property "sonar.verbose", true
        property "sonar.projectName", "Your project name"
        property "sonar.coverage.jacoco.xmlReportPaths", "${rootDir}/build/reports/jacoco/codeCoverageReport/codeCoverageReport.xml"
    }
}

Command to run test with coverage:

./gradlew codeCoverageReport
./gradlew sonarqube -x test (test is excluded since already run and sonarqube by default executes test)

Two things to be noted that made it work:

1. sonar.jacoco.reportPaths is deprecated. We need to use sonar.coverage.jacoco.xmlReportPaths. Check the documentation here
2. To make available sourcesets of all modules, looping over subprojects and accumulating sourcesets worked. subprojects.sourceSets.main.allSource.srcDirs did not work.

[bmuskalla]

We now added a sample showing how to properly configure a multi-module build using jacoco (HTML or XML for Sonar). As mentioned above already, xml is the preferred way to send code coverage information to Sonar. Another gotcha to look out for is that jacoco reports need to be generated before any of the sonar tasks run (see also SONARGRADL-68 and SONARGRADL-59

[cdpatelgit]

My Project configuration : Gradle 2.8 ,SonarQube 7.9
Problem Statement : Do we have any way to generate jacoco xml report for each subproject?
As my root project has so many subproject and we dont want to analyze all projects.
so in sonar-project.properties we have subproject level properties defined.
Can anyone please suggest how we can generate jacoco xml report for each subproject level?

[sekaiser]

@cdpatelgit could this Gradle plugin be useful to you? https://remal.gitlab.io/gradle-plugins/plugins/name.remal.merged-jacoco-report/

[ghost]

@ashwini-desai Thank you! I tried many ways to fix it only your fixes worked for me! Using gradle 6.5.1 and sonarqube 8.3.1.34397

[swatibajaj]

I am also trying to integrate jacoco with SonarQube for my multi-module project. I have an XML code coverage report generated. I have defined the following properties:

    property 'sonar.java.coveragePlugin', "jacoco"
    property "sonar.coverageReportPaths", file("${project.buildDir}/reports/jacoco")
    property "sonar.junit.reportPaths", file("${project.buildDir}/test-results/testDebugUnitTest/")
    property 'sonar.coverage.jacoco.xmlReportPaths', file("${project.buildDir}/reports/jacoco/jacocoFullReport.xml")
    property "sonar.it.reportPath", file("${project.buildDir}/reports/jacoco")

But I am unable to see the coverage report. Is there anything I am missing?

[rkrisztian]

Here's the solution I use with Gradle 7.4 (non-Android project), based on my earlier finding #8794 (comment) :

plugins {
	id 'jacoco'
}

repositories {
	// Need to resolve `org.jacoco.ant` jar on root level too. You can also use `allprojects {}` for common repo declarations.
	mavenCentral()
}

subprojects {
	pluginManager.withPlugin('java') {
		apply plugin: 'jacoco'
		tasks.withType(JacocoReport).configureEach { enabled = false }
		tasks.withType(Test).configureEach { finalizedBy ':jacocoMergedReport' }
	}
}

tasks.register('jacocoMergedReport', JacocoReport) {
	subprojects {
		pluginManager.withPlugin('jacoco') {
			sourceSets sourceSets.main
                        // `executionData tasks.withType(Test)` should work but it doesn't...
			executionData files(tasks.withType(Test)).filter { it.name.endsWith('.exec') && it.exists() }
		}
	}

	reports {
		xml.required = true
		html.required = true
	}
}

sonarqube {
	properties {
		property 'sonar.coverage.jacoco.xmlReportPaths', "${tasks.named('jacocoMergedReport', JacocoReport).get().reports.xml.outputLocation.get()}"
	}
}

subprojects {
	pluginManager.withPlugin('java') {
  	  	sonarqube {
		  	properties {
  				property 'sonar.coverage.jacoco.xmlReportPaths', null
	  		}
  		}
  	}
}

Currently I can't recommend the jacoco-report-aggregation plugin due its limitation that it cannot aggregate all test types: #23223

Update: See also more solution ideas at #26668

TL;DR (Why other approaches are problematic)

More explanation: The default approach is to enable JaCoCo in every project, but then JaCoCo itself will no longer track coverage across subprojects. So this is also a JaCoCo problem. One naive way to address this problem I saw was something like this:

subprojects {
    gradle.projectsEvaluated {
        tasks.withType(JacocoReport).configureEach {
            configurations.implementation.allDependencies.withType(ProjectDependency)*.dependencyProject.each {
                additionalSourceDirs files(it.sourceSets.main.java.srcDirs)
                additionalClassDirs files(it.sourceSets.main.output)
            }
        }
    }
}

But it has two problems: one is that now you can look for coverage for the same class in multiple HTML reports. And if you tell Sonar to track sources similarly, it will complain like:

File your-subproject/src/main/java/your/package/YourClass.java can't be indexed twice. Please check that inclusion/exclusion patterns produce disjoint sets for main and test files

You can fiddle around with the exclude patterns, but they will fail the same way if multiple subprojects depend on the same subproject. So the following configuration won't help every project, plus I think it's more complicated than having a single report:

subprojects {
    configurations.implementation.allDependencies.withType(ProjectDependency)*.dependencyProject.each { proj ->
        sonarqube {
            properties {
                property 'sonar.projectBaseDir', rootDir
                property 'sonar.sources', properties['sonar.sources'] + proj.sourceSets.main.java.srcDirs
                property 'sonar.tests', properties['sonar.tests'].split(/,/).collect { (projectDir as String) + '/' + it }
            }
        }
        proj.sonarqube {
            properties {
                property 'sonar.exclusions', ((properties['sonar.exclusions'] ?: []) +
                        proj.sourceSets.main.java.srcDirs.collect { "${it}/**" - "${proj.projectDir}/" }).unique()
            }
        }
    }
}

[himanshu2416]

I’ve not worked it out yet but has everything only done limited experiments. I suspect it is just a path issue. I plan to spend more time investigating next week and will update what I find.

hello @hajush , i am in a similar situation as once you were. For me we are using gradle 4.6 and our sonarqube server is 8.9.
We have a multi-project codebase and currently only a couple of those projects have coverage reports..but still i need to show the consolidated report for those projects on my sonarqube server...i believer we need to merge the xml file for each subproject report into a consolidated report...if you have a working solution can you please share it with me

[hajush]

Hi @himanshu2416 - unfortunately, I stopped working on that code base days after my post in 2020, so I had no time to do further investigations. I'm pretty sure that if you run the tasks.register('jacocoMergedReport', JacocoReport) as @rkrisztian described on April 4, 2023, you should get a merged report. It was working for me in 2020, I was just not sure why our coverage number declined so much, but it was probably a path or other configuration issue for the overall project.

[himanshu2416]

@hajush to be honest i did not try the approach by @rkrisztian as we are using gradle 4.6 and the task.register() method is available from gradle 5. But i will try something on similar lines. Thank you for the feedback.
@rkrisztian thank you for the approach.

[hajush]

@himanshu2416 I'm pretty sure we were using Gradle 6 at the time the beginning of 2020 and we definitely were using the task.register() method. Gradle is up to 8.1 today. Not sure anything will go well without upgrading, but if you do it should work.

[rkrisztian]

#23223 (comment) solves the issue with the aggregation plugin, which can be good if you need not just the full aggregation but also per test type.