Scoverage aggregation for multi-module project with module dependencies

[MathPlayer]

Hi!

I am porting from sbt to Mill a multi-module project and I have an issue related to scoverage aggregation.

Can someone have a look at the info below and tell me if I'm not doing some configuration or if there's something extra to be done by mill for the scoverage plugin?

Somehow, Mill doesn't consider transitive dependencies for aggregated coverage metrics.

I attached a minimalist project with both build systems configured: scoverage-demo.tar.gz

I'm pasting below excerpts from the README/build files for faster reading about the attached project (the README also contains commands to run to build / test / generate the coverage).

In short, there are two modules:

• foo (base module)
  • one class Foo (math methods)
  • one test class FooTest not covering any code from Foo.
• bar (depends on foo)
  • one class Bar (uses Foo math methods)
  • one test class BarTest, covering code from both Bar and Foo.

When all tests run and scoverage aggregation happens, foo should have the same coverage value with both sbt and mill.

The actual results:

• mill obtains 0% statement/branch coverage on foo
• sbt obtains 75% statement coverage, 50% branch coverage for foo

The build.mill content:

//| mvnDeps: ["com.lihaoyi::mill-contrib-scoverage:$MILL_VERSION"]

package build

import mill._
import scalalib._
import mill.contrib.scoverage.{ScoverageModule, ScoverageReport}

trait BaseModule extends ScalaModule with ScoverageModule {
  def scalaVersion = "2.13.18"
  def scoverageVersion = "2.5.2"
  
  trait Tests extends ScalaTests with ScoverageTests with TestModule.ScalaTest {
    def mvnDeps = Seq(mvn"org.scalatest::scalatest:3.2.20")
  }
}

object foo extends BaseModule {
  object test extends Tests
}

object bar extends BaseModule {
  def moduleDeps = Seq(foo)
  object test extends Tests
}

object scoverage extends ScoverageReport {
  override def scalaVersion = "2.13.18"
  override def scoverageVersion = "2.5.2"
}

The build.sbt content:

import sbt._
import Keys._

// Versions
val scalaV = "2.13.18"
val scalatestDep = "org.scalatest" %% "scalatest" % "3.2.20" % Test

// Generic project config
ThisBuild / scalaVersion := scalaV
ThisBuild / organization := "com.example"
ThisBuild / version := "0.1.0-SNAPSHOT"

// Scoverage
ThisBuild / coverageEnabled := true

lazy val commonSettings = Seq(
  scalaVersion := scalaV,
  libraryDependencies ++= Seq(scalatestDep)
)

// Foo module
lazy val foo = (project in file("foo"))
  .settings(commonSettings)
  .settings(
    name := "foo",
    Compile / scalaSource := baseDirectory.value / "src",
    Test / scalaSource := baseDirectory.value / "test" / "src"
  )

// Bar module that depends on foo
lazy val bar = (project in file("bar"))
  .settings(commonSettings)
  .settings(
    name := "bar",
    Compile / scalaSource := baseDirectory.value / "src",
    Test / scalaSource := baseDirectory.value / "test" / "src"
  )
  .dependsOn(foo)

// Root project
lazy val root = (project in file("."))
  .settings(
    name := "scoverage-demo",
    publish / skip := true
  )
  .aggregate(foo, bar)

addCommandAlias("cov", ";clean;coverage;test;coverageAggregate;")

[MathPlayer]

I think I found the surface of why this happens:

• ./mill show foo.scoverage.runClasspath lists out/foo/scoverage/compile.dest/classes
• ./mill show bar.scoverage.runClasspath lists out/foo/compile.dest/classes.

Should mill (via ScoverageModule) add all scoverage classes replace normal classes with scoverage classes in the run classpath for all module dependencies?
If yes, I can even look into a PR for it 😅

PS:

All looks fine with a brute alteration with the idea above in the demo build.mill (e.g. changing the runClasspath similar to what is done in ScoverageModule.scala).

object bar extends BaseModule {
  def moduleDeps = Seq(foo)
  object test extends Tests {
    override def runClasspath = Task {
      lazy val fooClasses = foo.compile().classes
      lazy val fooScoverageClasses = foo.scoverage.compile().classes

      super.runClasspath().map { path =>
        if (path == fooClasses) fooScoverageClasses else path
      }
    }

  }
}

I guess a generic idea should be done here, like get all scoverage classes from the whole tree of module deps and replace any compile ones with scoverage ones.