automatically provide dependencyOverrides for all scala-lang modules

[fommil]

if a project uses a particular version of scala, but a transitive dependency uses one of the scala-lang optionals (compiler, scalap, reflect, etc) then the older version is used. It would be good if something like this was automatic

    dependencyOverrides ++= Set(
      "org.scala-lang" % "scala-compiler" % scalaVersion.value,
      "org.scala-lang" % "scala-library" % scalaVersion.value,
      "org.scala-lang" % "scala-reflect" % scalaVersion.value,
      "org.scala-lang" % "scalap" % scalaVersion.value
    ),

[dwijnand]

Agreed.

On Tue, 24 Nov 2015 20:54 Sam Halliday notifications@github.com wrote:

if a project uses a particular version of scala, but a transitive
dependency uses one of the scala-lang optionals (compiler, scalap, reflect,
etc) then the older version is used. It would be good if something like
this was automatic

dependencyOverrides ++= Set(
  "org.scala-lang" % "scala-compiler" % scalaVersion.value,
  "org.scala-lang" % "scala-library" % scalaVersion.value,
  "org.scala-lang" % "scala-reflect" % scalaVersion.value,
  "org.scala-lang" % "scalap" % scalaVersion.value
),

—
Reply to this email directly or view it on GitHub
#2286.

[eed3si9n]

I don't know if the binary compatibility is guaranteed for using Scala compiler or scalap across minor releases.
Isn't it up to the build user to figure out what specific dependencies or overrides are needed?

[eed3si9n]

If json4s 3.2 depended on a particular version of scalap and if scalap changed the ABI, wouldn't this break json4s? (json4s 3.3 forked scalap to avoid this I think)

[paulp]

"Binary compatible" does not mean "compatible". They're supposed to be and probably are binary compatible. That still leaves an infinity of bugs to be discovered by mixing and matching jars from different scala versions.

[eed3si9n]

"Binary compatible" does not mean "compatible". They're supposed to be and probably are binary compatible. That still leaves an infinity of bugs to be discovered by mixing and matching jars from different scala versions.

I agree. The current law of the land on Maven/Ivy ecosystem is JAR swapping, and that's all we do.
I added eviction warning from similar concern (#1200) - http://www.scala-sbt.org/0.13/docs/Library-Management.html#Eviction+warning
Perhaps what we need is an option to turn that into an error so you are forced to resolve version conflicts manually. We can add Scala -library -compiler consistency check in there if the current imple of the eviction check doesn't catch it.

[paulp]

There are zero situations where you should mix jars from different releases. It ought to be possible to avoid adding any options, or at least an option which bears on this should be a lot more general.

[eed3si9n]

There are zero situations where you should mix jars from different releases. It ought to be possible to avoid adding any options, or at least an option which bears on this should be a lot more general.

1. How would I know what ModuleIDs belong to the the same "release"?
2. Is it actually true?

scala-library promises binary compatibility between the patches. Many projects knowingly or unknowingly bump up scala-library versions as long as they are within the same major version series such as 2.11. This should imply that the tie between scala-compiler and scala-library is not that tight either, even though it feels wrong.

On the other hand, libraries such as json4s 3.2.11 (https://repo1.maven.org/maven2/org/json4s/json4s-core_2.11/3.2.11/json4s-core_2.11-3.2.11.pom) makes use of scalap, which depends on scala compiler probably shouldn't be swapped since scala compiler I don't think makes promise of being swappable. json4s 3.2.11 depends on scalap 2.11.0 according to the pom. It's conceivable that someone wants to bump up scala-library to 2.11.7 for a bug fix etc but not want to bump up scala-compiler version to 2.11.7.

[paulp]

This should imply that ...

It doesn't. By happenstance it will usually work, just like code which is full of data races will usually work.

I have nothing to say about scalap, it is an uninteresting outlier. There are three jars of interest: library, reflect, compiler. Anyone who mixes these jars across versions is doing it wrong. That you can dream up multipronged situations like a bug fixed in the library while a new bug was introduced into the compiler and there is no workaround in either case is like planning around asteroid strikes. There is no reason to present this as yet another matter for individual configuration. Let the hypothetical people who need to override the behavior do so by performing some ridiculous hack of the sort I'm always having to do.

[ritschwumm]

@eed3si9n "that's all we do"?

i have this line in each and every build.sbt:
conflictManager := ConflictManager.strict

(for my own projects, i'd like to make it a bit less strict and accept changes in the minor version, but afaik that can't be done with sbt any more)

[eed3si9n]

@ritschwumm Right. I should have mentioned conflictManager := ConflictManager.strict, which was in my head writing above comments but may not have been obvious. conflictManager := ConflictManager.strict would bring forth the conflict of the diamond dependency problem to the build user. That would probably be the most rational solution. Given the probability of version conflict would increase as the dependency graph grows larger, I honestly don't know if that's a palatable solution. (Maybe it is)

The middle ground solution that I am seeking is automatically deducing which ties between the ModuleID is stronger given a version conflict. Maven picks the nearest. Ivy picks the latest (with some degree of Maven emulation for POMs). The popular opinion seems to be that scala toolchain is a special case.

One argument that I can make for that opinion is that scalaVersion is that sbt automatically retrieves the Scala compiler based on IvyScala, to compile the Scala source. This might motivate aligning "scalap", "scala-compiler", or "scala-reflect".

[milessabin]

Comment on #2634 would be most welcome.

[milessabin]

Fixed in #2634.