Mill doesn't evict older versions of transitive dependencies from moduleDeps

[lolgab]

Reproduction:

// build.sc
import mill._
import mill.scalalib._

object bar extends ScalaModule {
  def scalaVersion = "2.13.12"
  def moduleDeps = Seq(foo)
  def ivyDeps = super.ivyDeps() ++ Agg(
    ivy"org.typelevel::cats-core::2.9.0",
  )
}
object foo extends ScalaModule {
  def scalaVersion = "2.13.12"
  def ivyDeps = super.ivyDeps() ++ Agg(
    ivy"io.circe::circe-core::0.14.0"
  )
}

classpath with Mill 0.11.2 (note the two artifacts of cats-core_2.13):

> mill show bar.compileClasspath
[
  "ref:v0:c984eca8:/Users/lorenzo/scala/repro/foo/compile-resources",
  "qref:v1:19c63377:/Users/lorenzo/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/io/circe/circe-core_2.13/0.14.0/circe-core_2.13-0.14.0.jar",
  "qref:v1:0004e741:/Users/lorenzo/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/scala-lang/scala-library/2.13.12/scala-library-2.13.12.jar",
  "qref:v1:03c52e9e:/Users/lorenzo/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/io/circe/circe-numbers_2.13/0.14.0/circe-numbers_2.13-0.14.0.jar",
  "qref:v1:59fac2bc:/Users/lorenzo/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/typelevel/cats-core_2.13/2.6.1/cats-core_2.13-2.6.1.jar",
  "qref:v1:b856e496:/Users/lorenzo/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/typelevel/cats-kernel_2.13/2.6.1/cats-kernel_2.13-2.6.1.jar",
  "qref:v1:cfb647ca:/Users/lorenzo/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/typelevel/simulacrum-scalafix-annotations_2.13/0.5.4/simulacrum-scalafix-annotations_2.13-0.5.4.jar",
  "ref:v0:c984eca8:/Users/lorenzo/scala/repro/out/foo/compile.dest/classes",
  "ref:v0:c984eca8:/Users/lorenzo/scala/repro/bar/compile-resources",
  "qref:v1:f482fc22:/Users/lorenzo/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/typelevel/cats-core_2.13/2.9.0/cats-core_2.13-2.9.0.jar",
  "qref:v1:045a4a25:/Users/lorenzo/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/typelevel/cats-kernel_2.13/2.9.0/cats-kernel_2.13-2.9.0.jar"
]

classpath with Mill 0.10.12 (which has a single artifact of cats-core_2.13):

> mill show bar.compileClasspath
[
  "ref:c984eca8:/Users/lorenzo/scala/repro/foo/resources",
  "ref:c984eca8:/Users/lorenzo/scala/repro/out/foo/compile.dest/classes",
  "ref:c984eca8:/Users/lorenzo/scala/repro/bar/resources",
  "qref:f482fc22:/Users/lorenzo/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/typelevel/cats-core_2.13/2.9.0/cats-core_2.13-2.9.0.jar",
  "qref:0004e741:/Users/lorenzo/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/scala-lang/scala-library/2.13.12/scala-library-2.13.12.jar",
  "qref:19c63377:/Users/lorenzo/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/io/circe/circe-core_2.13/0.14.0/circe-core_2.13-0.14.0.jar",
  "qref:045a4a25:/Users/lorenzo/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/typelevel/cats-kernel_2.13/2.9.0/cats-kernel_2.13-2.9.0.jar",
  "qref:03c52e9e:/Users/lorenzo/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/io/circe/circe-numbers_2.13/0.14.0/circe-numbers_2.13-0.14.0.jar"
]

[lefou]

Looks like an regression, probably introduced in #2425. The culprit seems to be in this code

mill/scalalib/src/mill/scalalib/JavaModule.scala

Lines 222 to 227 in 32f9479

	def transitiveCompileClasspath: T[Agg[PathRef]] = T {
	T.traverse(
	(moduleDeps ++ compileModuleDeps).flatMap(_.transitiveModuleDeps).distinct
	)(m => T.task { m.compileClasspath() ++ Agg(m.compile().classes) })()
	.flatten
	}

When traversing modules, we should only collect local resources, but not include already resolved ivyDeps, as we want to resolve all transitive ivyDeps later consistently. But the included compileClasspath might already contain resolved ivyDeps. That way, we end up with multiple resolved versions of the same dependency.