New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

sbt exclusion rules don't handle Scala version specific dependencies #1518

Closed
tlockney opened this Issue Aug 8, 2014 · 27 comments

Comments

Projects
None yet
@tlockney
Contributor

tlockney commented Aug 8, 2014

steps

"net.liftweb" %% "lift-mapper" % "2.6-M4" % "compile" excludeAll(ExclusionRule("net.liftweb", "lift-json"))

problem

"net.liftweb" %% "lift-json" is actually not excluded (because you need cross versioned "lift-json_2.10").

expectation

ExclusionRule could smart up some how, or accept GroupArtifactID. The LHS of excludeAll is ModuleID so it should already know the crossVersion mode that its in.

original report

This came up as an issue recently, referenced in this Stack Overflow post from Channing Walton. Basically, the issue is that since exclusion rules are a pretty direct mapping (as I recall, anyway) onto Ivy, they don't include relevant information that sbt knows, such as the Scala version being referenced. So, while you can include a dependency that's Scala specific using the %% syntax, there's no similar mapping available for exclusions. I'm not sure what form a solution to this might take, but at least extending it to include something along the lines of the ModuleID support for %% might make sense, as a start.

@eed3si9n eed3si9n added the Enhancement label Aug 9, 2014

@eed3si9n

This comment has been minimized.

Show comment
Hide comment
@eed3si9n

eed3si9n Aug 9, 2014

Member

Thanks for reporting this. I've ran into this myself while trying to use exclusion rules, and it's annoying.

Member

eed3si9n commented Aug 9, 2014

Thanks for reporting this. I've ran into this myself while trying to use exclusion rules, and it's annoying.

@eed3si9n

This comment has been minimized.

Show comment
Hide comment
@eed3si9n

eed3si9n Aug 9, 2014

Member

scalaFullVersion is available during resolution via IvyScala instance on moduleSetting passed in to create IvySbt#Module. Using this we probably could transform the exclusion rules, which feels a bit evil. Maybe it's better to use GroupArtifactID ("org" %% "name").

Member

eed3si9n commented Aug 9, 2014

scalaFullVersion is available during resolution via IvyScala instance on moduleSetting passed in to create IvySbt#Module. Using this we probably could transform the exclusion rules, which feels a bit evil. Maybe it's better to use GroupArtifactID ("org" %% "name").

@mosesn

This comment has been minimized.

Show comment
Hide comment
@mosesn

mosesn Aug 27, 2014

👍 exclude is very difficult to use against scala jars right now, this would be a significant improvement.

mosesn commented Aug 27, 2014

👍 exclude is very difficult to use against scala jars right now, this would be a significant improvement.

@dpratt

This comment has been minimized.

Show comment
Hide comment
@dpratt

dpratt Nov 22, 2014

Contributor

I vote for this as well - it'd be nice to have syntax that would look like

"com.typesafe.play" %% "play-json" % "2.3.6" exclude("com.typesafe.play" %% "play-iteratees")

Contributor

dpratt commented Nov 22, 2014

I vote for this as well - it'd be nice to have syntax that would look like

"com.typesafe.play" %% "play-json" % "2.3.6" exclude("com.typesafe.play" %% "play-iteratees")

@magro

This comment has been minimized.

Show comment
Hide comment
@magro

magro Jan 4, 2015

+1, for now I'm using a very simple and stupid solution: exclude("com.typesafe.play", "play_" + scalaVersion.value.substring(0, 4)

magro commented Jan 4, 2015

+1, for now I'm using a very simple and stupid solution: exclude("com.typesafe.play", "play_" + scalaVersion.value.substring(0, 4)

@reid-spencer

This comment has been minimized.

Show comment
Hide comment
@reid-spencer

reid-spencer commented Jan 7, 2015

+1

@eed3si9n

This comment has been minimized.

Show comment
Hide comment
@eed3si9n

eed3si9n Jan 7, 2015

Member

I did add project-level exclusion that uses ModuleID syntax in #1748

Member

eed3si9n commented Jan 7, 2015

I did add project-level exclusion that uses ModuleID syntax in #1748

@eed3si9n eed3si9n added this to the 0.13.8 milestone Jan 7, 2015

@eed3si9n eed3si9n removed this from the 0.13.8 milestone Mar 26, 2015

@stephanh

This comment has been minimized.

Show comment
Hide comment
@stephanh

stephanh May 1, 2015

+1 for a version of exclude that takes into consideration the scala binary version.

stephanh commented May 1, 2015

+1 for a version of exclude that takes into consideration the scala binary version.

@LouisJB

This comment has been minimized.

Show comment
Hide comment
@LouisJB

LouisJB May 7, 2015

+1 (+100) - really would be very useful indeed...

LouisJB commented May 7, 2015

+1 (+100) - really would be very useful indeed...

@dnrusakov

This comment has been minimized.

Show comment
Hide comment
@dnrusakov

dnrusakov commented Mar 25, 2016

+1

@mwegrz

This comment has been minimized.

Show comment
Hide comment
@mwegrz

mwegrz commented Mar 25, 2016

+1

@tlockney

This comment has been minimized.

Show comment
Hide comment
@tlockney

tlockney Mar 25, 2016

Contributor

This actually just bit me again. Might take a look in the near future at adding a fix for this, if time permits.

Contributor

tlockney commented Mar 25, 2016

This actually just bit me again. Might take a look in the near future at adding a fix for this, if time permits.

@noahlz

This comment has been minimized.

Show comment
Hide comment
@noahlz

noahlz commented Jul 22, 2016

👍

@mziemba

This comment has been minimized.

Show comment
Hide comment
@mziemba

mziemba commented Aug 5, 2016

+1

@pkelly03

This comment has been minimized.

Show comment
Hide comment
@pkelly03

pkelly03 commented Oct 13, 2016

👍

@edesdan

This comment has been minimized.

Show comment
Hide comment
@edesdan

edesdan commented Oct 20, 2016

+1

jonbuffington added a commit to fuseelements/spark-jobserver that referenced this issue Nov 5, 2016

fix: exclude org.jboss.netty classes
The akka-remote transitive dependency brings an older io.netty jar
into the assembly causing an occasional Java verifier exception when
starting SJS.

See <sbt/sbt#1518> why the simple exclude of
akka-remote does not work.

jonbuffington added a commit to fuseelements/spark-jobserver that referenced this issue Nov 5, 2016

fix: exclude org.jboss.netty classes
The akka-remote transitive dependency brings an older io.netty jar
into the assembly causing an occasional Java verifier exception when
starting SJS.

See <sbt/sbt#1518> why the simple exclude of
akka-remote does not work.

jonbuffington added a commit to fuseelements/spark-jobserver that referenced this issue Nov 5, 2016

fix: exclude old org.jboss.netty classes
The akka-remote transitive dependency brings an older io.netty jar into
the assembly causing an occasional Java verifier exception starting SJS.

See <sbt/sbt#1518> why the simple exclude of
akka-remote does not work.

jonbuffington added a commit to fuseelements/spark-jobserver that referenced this issue Nov 5, 2016

fix: exclude old org.jboss.netty classes
The akka-remote transitive dependency brings an older io.netty jar into
the assembly causing an occasional Java verifier exception starting SJS.

See <sbt/sbt#1518> why the simple exclude of
akka-remote does not work.

jonbuffington added a commit to fuseelements/spark-jobserver that referenced this issue Nov 5, 2016

fix: exclude old org.jboss.netty classes
The akka-remote transitive dependency brings an older io.netty jar into
the assembly causing an occasional "Wrong return type in function"
exception starting SJS:

```
[sparkDriver-akka.actor.default-dispatcher-2] ERROR akka.actor.ActorSystemImpl - Uncaught fatal error from thread [sparkDriver-akka.remote.default-remote-dispatcher-5] shutting down ActorSystem [sparkDriver]
java.lang.VerifyError: (class: org/jboss/netty/channel/socket/nio/NioWorkerPool, method: createWorker signature: (Ljava/util/concurrent/Executor;)Lorg/jboss/netty/channel/socket/nio/AbstractNioWorker;) Wrong return type in function
```

See <sbt/sbt#1518> why the simple exclude of
akka-remote does not work.

jonbuffington added a commit to fuseelements/spark-jobserver that referenced this issue Nov 5, 2016

fix: exclude old org.jboss.netty classes
The akka-remote transitive dependency brings an older io.netty jar into
the assembly causing an occasional "Wrong return type in function"
exception starting SJS:

```
[sparkDriver-akka.actor.default-dispatcher-2] ERROR akka.actor.ActorSystemImpl - Uncaught fatal error from thread [sparkDriver-akka.remote.default-remote-dispatcher-5] shutting down ActorSystem [sparkDriver] java.lang.VerifyError: (class: org/jboss/netty/channel/socket/nio/NioWorkerPool, method: createWorker signature: (Ljava/util/concurrent/Executor;)Lorg/jboss/netty/channel/socket/nio/AbstractNioWorker;) Wrong return type in function
```

See <sbt/sbt#1518> why the simple exclude of
akka-remote does not work.
@ryan-williams

This comment has been minimized.

Show comment
Hide comment
@ryan-williams

ryan-williams Dec 4, 2016

I'm using this workaround:

"com.holdenkarau" %% "spark-testing-base" % "1.6.3_0.4.7" exclude("org.scalatest", s"scalatest_${scalaBinaryVersion.value}")

If anyone has better/cleaner ideas I'd love to hear them!

ryan-williams commented Dec 4, 2016

I'm using this workaround:

"com.holdenkarau" %% "spark-testing-base" % "1.6.3_0.4.7" exclude("org.scalatest", s"scalatest_${scalaBinaryVersion.value}")

If anyone has better/cleaner ideas I'd love to hear them!

@domdorn

This comment has been minimized.

Show comment
Hide comment
@domdorn

domdorn Apr 4, 2017

hmm.. any updates? was affected by this just today.. >2 years since ticket was opened..

domdorn commented Apr 4, 2017

hmm.. any updates? was affected by this just today.. >2 years since ticket was opened..

@arpadtamasi

This comment has been minimized.

Show comment
Hide comment
@arpadtamasi

arpadtamasi Apr 12, 2017

Annoying me, too

arpadtamasi commented Apr 12, 2017

Annoying me, too

@manjuraj

This comment has been minimized.

Show comment
Hide comment
@manjuraj

manjuraj commented Apr 20, 2017

+1

@aakashsa

This comment has been minimized.

Show comment
Hide comment
@aakashsa

aakashsa commented Apr 20, 2017

+1

jvican added a commit to scalacenter/librarymanagement that referenced this issue Apr 26, 2017

Fix sbt/sbt#1518: Handle cross-versioned exclusions
The following PR does two things:

* Removes the unnecessary `SbtExclusionRule` that was introduced to
  exclude artifacts at the project level (and not the dependency level).
  This change was done in an independent class to avoid breaking
  bincompat in 0.13.x series.
* Handle exclusion rules correctly, propagating the cross version to the
  exclusions of the dependencies.

To fix sbt/sbt#1518, this PR takes the avenue taken in
`SbtExclusionRule`, it accepts `GroupArtifactID` which should be the
preferred way to specify dependencies from now on. Unlike
`SbtExclusionRule`, it also supports `ModuleID` for those that want to
exclude a concrete dependency.

`InclExcl` did not have any tests. The following commit also adds a
testing suite for it, showing how the issue it's fixed and how you
should use `exclude` if you're calling directly `ExclusionRule` instead
of passing in `GroupArtifactID` and `ModuleID`.
@jvican

This comment has been minimized.

Show comment
Hide comment
@jvican

jvican Apr 26, 2017

Member

Hello everyone and thank you for your patience!

The Scala Center has contributed a fix for this here: sbt/librarymanagement#88. It will be merged in 1.0. That PR depends on a big pending PR reformatting all the sources, so don't get scared when you see the diff. The actual fix is this commit and these are the tests that prove the issue is fixed.

Member

jvican commented Apr 26, 2017

Hello everyone and thank you for your patience!

The Scala Center has contributed a fix for this here: sbt/librarymanagement#88. It will be merged in 1.0. That PR depends on a big pending PR reformatting all the sources, so don't get scared when you see the diff. The actual fix is this commit and these are the tests that prove the issue is fixed.

@tlockney

This comment has been minimized.

Show comment
Hide comment
@tlockney

tlockney Apr 26, 2017

Contributor

@jvican that's great news! Very much looking forward to seeing this commonly encountered, but tricky to recognize scenario getting resolved. Makes me all the more excited to see 1.0 come to life!

Contributor

tlockney commented Apr 26, 2017

@jvican that's great news! Very much looking forward to seeing this commonly encountered, but tricky to recognize scenario getting resolved. Makes me all the more excited to see 1.0 come to life!

jvican added a commit to scalacenter/librarymanagement that referenced this issue Apr 26, 2017

Fix sbt/sbt#1518: Handle cross-versioned exclusions
The following PR does two things:

* Removes the unnecessary `SbtExclusionRule` that was introduced to
  exclude artifacts at the project level (and not the dependency level).
  This change was done in an independent class to avoid breaking
  bincompat in 0.13.x series.
* Handle exclusion rules correctly, propagating the cross version to the
  exclusions of the dependencies.

To fix sbt/sbt#1518, this PR takes the avenue taken in
`SbtExclusionRule`, it accepts `GroupArtifactID` which should be the
preferred way to specify dependencies from now on. Unlike
`SbtExclusionRule`, it also supports `ModuleID` for those that want to
exclude a concrete dependency.

`InclExcl` did not have any tests. The following commit also adds a
testing suite for it, showing how the issue it's fixed and how you
should use `exclude` if you're calling directly `ExclusionRule` instead
of passing in `GroupArtifactID` and `ModuleID`.

jvican added a commit to scalacenter/librarymanagement that referenced this issue Apr 26, 2017

Fix sbt/sbt#1518: Handle cross-versioned exclusions
The following PR does two things:

* Removes the unnecessary `SbtExclusionRule` that was introduced to
  exclude artifacts at the project level (and not the dependency level).
  This change was done in an independent class to avoid breaking
  bincompat in 0.13.x series.
* Handle exclusion rules correctly, propagating the cross version to the
  exclusions of the dependencies.

To fix sbt/sbt#1518, this PR takes the avenue taken in
`SbtExclusionRule`, it accepts `GroupArtifactID` which should be the
preferred way to specify dependencies from now on. Unlike
`SbtExclusionRule`, it also supports `ModuleID` for those that want to
exclude a concrete dependency.

`InclExcl` did not have any tests. The following commit also adds a
testing suite for it, showing how the issue it's fixed and how you
should use `exclude` if you're calling directly `ExclusionRule` instead
of passing in `GroupArtifactID` and `ModuleID`.

eed3si9n added a commit to sbt/librarymanagement that referenced this issue May 2, 2017

Merge pull request #88 from scalacenter/issue/1518
Fix sbt/sbt#1518: Handle cross-versioned exclusions
@samudurand

This comment has been minimized.

Show comment
Hide comment
@samudurand

samudurand Sep 22, 2017

I am still having the issue related to the number of version in the exclusion
This is not excluding :

  val scalaKafkaClient = "net.cakesolutions" %% "scala-kafka-client-akka" % scalaKafkaClientVersion exclude ("com.typesafe.akka", "akka-actor")

But this works

  val scalaKafkaClient = "net.cakesolutions" %% "scala-kafka-client-akka" % scalaKafkaClientVersion exclude ("com.typesafe.akka", "akka-actor_12")

I thought from sbt 1.0 I would not need to explicitely set the scala version ?

samudurand commented Sep 22, 2017

I am still having the issue related to the number of version in the exclusion
This is not excluding :

  val scalaKafkaClient = "net.cakesolutions" %% "scala-kafka-client-akka" % scalaKafkaClientVersion exclude ("com.typesafe.akka", "akka-actor")

But this works

  val scalaKafkaClient = "net.cakesolutions" %% "scala-kafka-client-akka" % scalaKafkaClientVersion exclude ("com.typesafe.akka", "akka-actor_12")

I thought from sbt 1.0 I would not need to explicitely set the scala version ?

@jvican

This comment has been minimized.

Show comment
Hide comment
@jvican

jvican Sep 25, 2017

Member

@samudurand

val scalaKafkaClient = "net.cakesolutions" %% "scala-kafka-client-akka" % scalaKafkaClientVersion exclude ("com.typesafe.akka" %% "akka-actor")
Member

jvican commented Sep 25, 2017

@samudurand

val scalaKafkaClient = "net.cakesolutions" %% "scala-kafka-client-akka" % scalaKafkaClientVersion exclude ("com.typesafe.akka" %% "akka-actor")
@samudurand

This comment has been minimized.

Show comment
Hide comment
@samudurand

samudurand Sep 25, 2017

@jvican looks good ! I will try that tonight

samudurand commented Sep 25, 2017

@jvican looks good ! I will try that tonight

@samudurand

This comment has been minimized.

Show comment
Hide comment
@samudurand

samudurand Sep 28, 2017

@jvican this does work but not with exclude , instead use excludeAll()

samudurand commented Sep 28, 2017

@jvican this does work but not with exclude , instead use excludeAll()

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment