New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cross-version issue with Scala 2.12.0-RC1 #485

Closed
hseeberger opened this Issue Oct 2, 2016 · 25 comments

Comments

Projects
None yet
8 participants
@hseeberger

hseeberger commented Oct 2, 2016

[info] Updating {file:/Users/heiko/projects/scala/new-in-scala-212/}new-in-scala-212...
[warn] Binary version (2.11) for dependency org.scala-lang#scala-library;2.11.8
[warn]  in de.heikoseeberger#new-in-scala-212_2.12.0-RC1;f023808b8f857706a6c3d4d3a633496683cf471b-SNAPSHOT differs from Scala binary version in project (2.12.0-RC1).
[info] Resolving com.github.scopt#scopt_2.11;3.3.0 ...
[info] Done updating.
[error] Modules were resolved with conflicting cross-version suffixes in {file:/Users/heiko/projects/scala/new-in-scala-212/}new-in-scala-212:
[error]    org.scala-lang.modules:scala-xml _2.11, _2.12.0-RC1
[trace] Stack trace suppressed: run last *:update for the full output.
[error] (*:update) Conflicting cross-version suffixes in: org.scala-lang.modules:scala-xml
[error] Total time: 0 s, completed Oct 2, 2016 3:39:12 PM

@olafurpg olafurpg added the sbt label Oct 3, 2016

@olafurpg

This comment has been minimized.

Show comment
Hide comment
@olafurpg

olafurpg Oct 3, 2016

Member

Hmmm. I'm not sure what's causing this problem. The sbt plugin adds dependencies in the "scalafmt" scope so it should not conflict with other compile/runtime dependencies.

A good place to start debugging this might be here: https://github.com/olafurpg/scalafmt/blob/510534c3677d1cd6e27699601374654ebdba85c6/scalafmtSbt/src/main/scala/org/scalafmt/sbt/ScalaFmtPlugin.scala#L93-L94

Member

olafurpg commented Oct 3, 2016

Hmmm. I'm not sure what's causing this problem. The sbt plugin adds dependencies in the "scalafmt" scope so it should not conflict with other compile/runtime dependencies.

A good place to start debugging this might be here: https://github.com/olafurpg/scalafmt/blob/510534c3677d1cd6e27699601374654ebdba85c6/scalafmtSbt/src/main/scala/org/scalafmt/sbt/ScalaFmtPlugin.scala#L93-L94

@LiMuBei

This comment has been minimized.

Show comment
Hide comment
@LiMuBei

LiMuBei Oct 10, 2016

I'm seeing the same issue. My projects use Scala 2.10.6 and scalafmt sbt plugin pulls in 2.11 which causes the warnings.

LiMuBei commented Oct 10, 2016

I'm seeing the same issue. My projects use Scala 2.10.6 and scalafmt sbt plugin pulls in 2.11 which causes the warnings.

@olafurpg

This comment has been minimized.

Show comment
Hide comment
@olafurpg

olafurpg Oct 11, 2016

Member

@LiMuBei or @hseeberger Could you provide a minimal repo to reproduce the bug?

Member

olafurpg commented Oct 11, 2016

@LiMuBei or @hseeberger Could you provide a minimal repo to reproduce the bug?

@hseeberger

This comment has been minimized.

Show comment
Hide comment
@hseeberger

hseeberger Oct 11, 2016

Extract issue485.zip, cd into the issue485 directory, start sbt and execute compile or scalafmt.

hseeberger commented Oct 11, 2016

Extract issue485.zip, cd into the issue485 directory, start sbt and execute compile or scalafmt.

@olafurpg

This comment has been minimized.

Show comment
Hide comment
@olafurpg

olafurpg Oct 11, 2016

Member

Thanks. I was able to produce the error

java.lang.NoClassDefFoundError: scala/Product$class
    at org.scalafmt.config.Docstrings$ScalaDoc$.<init>(Docstrings.scala:8)
    at org.scalafmt.config.Docstrings$ScalaDoc$.<clinit>(Docstrings.scala)
    at org.scalafmt.config.ScalafmtConfig$.apply$default$2(ScalafmtConfig.scala:2)
    at org.scalafmt.config.Settings$class.$init$(Settings.scala:17)
    at org.sc
...

which indicates that scala-library is missing from the classpath. We had a similar issue in #190. However, running

> show issue485/libraryDependencies
[info] List(org.scala-lang:scala-library:2.12.0-RC1, org.scala-lang:scala-library:2.11.8:scalafmt, com.geirsson:scalafmt-cli_2.11:0.4.4:scalafmt, org.scalatest:scalatest:3.0.0:test)

shows that scala-library:2.11.8 is in fact a dependency to the scalafmt configuration. I don't know what to do, it may be that SBT is special casing scala-library for some weird reason.

Do you only get the following error?

[error] Modules were resolved with conflicting cross-version suffixes in {file:/Users/heiko/projects/scala/new-in-scala-212/}new-in-scala-212:
[error]    org.scala-lang.modules:scala-xml _2.11, _2.12.0-RC1
Member

olafurpg commented Oct 11, 2016

Thanks. I was able to produce the error

java.lang.NoClassDefFoundError: scala/Product$class
    at org.scalafmt.config.Docstrings$ScalaDoc$.<init>(Docstrings.scala:8)
    at org.scalafmt.config.Docstrings$ScalaDoc$.<clinit>(Docstrings.scala)
    at org.scalafmt.config.ScalafmtConfig$.apply$default$2(ScalafmtConfig.scala:2)
    at org.scalafmt.config.Settings$class.$init$(Settings.scala:17)
    at org.sc
...

which indicates that scala-library is missing from the classpath. We had a similar issue in #190. However, running

> show issue485/libraryDependencies
[info] List(org.scala-lang:scala-library:2.12.0-RC1, org.scala-lang:scala-library:2.11.8:scalafmt, com.geirsson:scalafmt-cli_2.11:0.4.4:scalafmt, org.scalatest:scalatest:3.0.0:test)

shows that scala-library:2.11.8 is in fact a dependency to the scalafmt configuration. I don't know what to do, it may be that SBT is special casing scala-library for some weird reason.

Do you only get the following error?

[error] Modules were resolved with conflicting cross-version suffixes in {file:/Users/heiko/projects/scala/new-in-scala-212/}new-in-scala-212:
[error]    org.scala-lang.modules:scala-xml _2.11, _2.12.0-RC1
@hseeberger

This comment has been minimized.

Show comment
Hide comment
@hseeberger

hseeberger Oct 11, 2016

Yes, I can only see the latter.

Notice that the Java classpath is linear and above 2.12 comes before 2.11.

hseeberger commented Oct 11, 2016

Yes, I can only see the latter.

Notice that the Java classpath is linear and above 2.12 comes before 2.11.

@olafurpg

This comment has been minimized.

Show comment
Hide comment
@olafurpg

olafurpg Oct 11, 2016

Member

Notice that the Java classpath is linear and above 2.12 comes before 2.11.

Good point. This could be the reason why it currently works with 2.10 and not 2.12.

Member

olafurpg commented Oct 11, 2016

Notice that the Java classpath is linear and above 2.12 comes before 2.11.

Good point. This could be the reason why it currently works with 2.10 and not 2.12.

@hseeberger

This comment has been minimized.

Show comment
Hide comment
@hseeberger

hseeberger Oct 11, 2016

I think it's plain wrong to have two versions of Scala (any lib) on the classpath.

Von meinem iPhone gesendet

Am 11.10.2016 um 16:08 schrieb Ólafur Páll Geirsson notifications@github.com:

Notice that the Java classpath is linear and above 2.12 comes before 2.11.

Good point. This could be the reason why it currently works with 2.10 and not 2.12.


You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

hseeberger commented Oct 11, 2016

I think it's plain wrong to have two versions of Scala (any lib) on the classpath.

Von meinem iPhone gesendet

Am 11.10.2016 um 16:08 schrieb Ólafur Páll Geirsson notifications@github.com:

Notice that the Java classpath is linear and above 2.12 comes before 2.11.

Good point. This could be the reason why it currently works with 2.10 and not 2.12.


You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

@olafurpg

This comment has been minimized.

Show comment
Hide comment
@olafurpg

olafurpg Oct 11, 2016

Member

Those libraries are in a scalafmt configuration just like test libraries are in the test configuration (i.e., % "test"). There should be no conflict in the classpath. It appears SBT is special handling scala-library for some reason.

Member

olafurpg commented Oct 11, 2016

Those libraries are in a scalafmt configuration just like test libraries are in the test configuration (i.e., % "test"). There should be no conflict in the classpath. It appears SBT is special handling scala-library for some reason.

@SethTisue

This comment has been minimized.

Show comment
Hide comment
@SethTisue

SethTisue Oct 13, 2016

why is the plugin adding anything at all to my project's libraryDependencies? shouldn't scalafmt stuff be a dependency of my sbt build definition only?

SethTisue commented Oct 13, 2016

why is the plugin adding anything at all to my project's libraryDependencies? shouldn't scalafmt stuff be a dependency of my sbt build definition only?

@olafurpg

This comment has been minimized.

Show comment
Hide comment
@olafurpg

olafurpg Oct 13, 2016

Member

SBT plugins must build against 2.10 and scala.meta (which scalafmt depends on) is 2.11 only. To work around this SBT limitation the plugin classloads scalafmt with scala-library 2.11.8 through a hidden scalafmt configuration in libraryDependencies. This seems to work great for most builds but sometimes causes troubles like in this issue and #190.

Here's a more detailed explanation: https://gitter.im/olafurpg/scalafmt?at=573a3d44c61823687d3c6cd3

Here is where the plugin classloads: https://github.com/olafurpg/scalafmt/blob/b6cc4e0919be43516c41746be3984db598b950b6/scalafmtSbt/src/main/scala/org/scalafmt/sbt/ScalaFmtPlugin.scala#L137

I would love any help to improve the SBT plugin. I personally don't use the SBT plugin so my motivation to invest more of my free time in fixing this issue is low.

It appears many people (and large organizations) rely on the plugin so there must be someone who's willing to lend an extra hand. I have asked for help on the gitter channel but unfortunately got no response.

I have scripted tests so the best approach to fix this issue is to reproduce the issue in the sbt-test folder (see issue485.zip above) and start poking at ScalafmtPlugin.

Member

olafurpg commented Oct 13, 2016

SBT plugins must build against 2.10 and scala.meta (which scalafmt depends on) is 2.11 only. To work around this SBT limitation the plugin classloads scalafmt with scala-library 2.11.8 through a hidden scalafmt configuration in libraryDependencies. This seems to work great for most builds but sometimes causes troubles like in this issue and #190.

Here's a more detailed explanation: https://gitter.im/olafurpg/scalafmt?at=573a3d44c61823687d3c6cd3

Here is where the plugin classloads: https://github.com/olafurpg/scalafmt/blob/b6cc4e0919be43516c41746be3984db598b950b6/scalafmtSbt/src/main/scala/org/scalafmt/sbt/ScalaFmtPlugin.scala#L137

I would love any help to improve the SBT plugin. I personally don't use the SBT plugin so my motivation to invest more of my free time in fixing this issue is low.

It appears many people (and large organizations) rely on the plugin so there must be someone who's willing to lend an extra hand. I have asked for help on the gitter channel but unfortunately got no response.

I have scripted tests so the best approach to fix this issue is to reproduce the issue in the sbt-test folder (see issue485.zip above) and start poking at ScalafmtPlugin.

@mdedetrich

This comment has been minimized.

Show comment
Hide comment
@mdedetrich

mdedetrich Oct 13, 2016

@olafurpg Can a combination of proguard + shading work? You could theoretically package scalafmt + scala 2.10 and make scalafmt implement some java interface (which the shade plugin wouldn't rename) that you can interface through via sbt.

Its probably a lot of work though

mdedetrich commented Oct 13, 2016

@olafurpg Can a combination of proguard + shading work? You could theoretically package scalafmt + scala 2.10 and make scalafmt implement some java interface (which the shade plugin wouldn't rename) that you can interface through via sbt.

Its probably a lot of work though

@olafurpg

This comment has been minimized.

Show comment
Hide comment
@olafurpg

olafurpg Oct 14, 2016

Member

@mdedetrich I don't know enough about proguard to make an intelligent guess.

I tracked down the issue to the fact that the resolver evicts scala-library 2.11.8 in favor of 2.12 even though the 2.11.8 dependency is in a separate configuration. Here's the report:

[info]      - 2.11.8
[info]          status: release
[info]          publicationDate: Fri Mar 04 16:26:33 CET 2016
[info]          resolver: sbt-chain
[info]          artifactResolver: sbt-chain
[info]          evicted: true
[info]          evictedData: latest-revision
[info]          homepage: http://www.scala-lang.org/
[info]          textraAttributes: Map(info.apiURL -> http://www.scala-lang.org/api/2.11.8/)
[info]          isDefault: false
[info]          configurations: default, compile, runtime, default(compile), master
[info]          licenses: (BSD 3-Clause,Some(http://www.scala-lang.org/license.html))
[info]          callers: de.heikoseeberger:issue485_2.12.0-RC1:20161014T174625

I tried

dependencyOverrides += "org.scala-lang" % "scala-library" % org.scalafmt.Versions.scala % "scalafmt"
// and
libraryDependencies += "org.scala-lang" % "scala-library" % org.scalafmt.Versions.scala % "scalafmt" force()

with no luck. It appears sbt is special handling scala-library. I did find a way to solve this issue however,

libraryDependencies += "org.typelevel" % "scala-library"     % org.scalafmt.Versions.scala   % "scalafmt"

It seems by changing the organisation to org.typelevel then sbt does not special case scala-library anymore. Any better suggestions? cc/ @dwijnand

Member

olafurpg commented Oct 14, 2016

@mdedetrich I don't know enough about proguard to make an intelligent guess.

I tracked down the issue to the fact that the resolver evicts scala-library 2.11.8 in favor of 2.12 even though the 2.11.8 dependency is in a separate configuration. Here's the report:

[info]      - 2.11.8
[info]          status: release
[info]          publicationDate: Fri Mar 04 16:26:33 CET 2016
[info]          resolver: sbt-chain
[info]          artifactResolver: sbt-chain
[info]          evicted: true
[info]          evictedData: latest-revision
[info]          homepage: http://www.scala-lang.org/
[info]          textraAttributes: Map(info.apiURL -> http://www.scala-lang.org/api/2.11.8/)
[info]          isDefault: false
[info]          configurations: default, compile, runtime, default(compile), master
[info]          licenses: (BSD 3-Clause,Some(http://www.scala-lang.org/license.html))
[info]          callers: de.heikoseeberger:issue485_2.12.0-RC1:20161014T174625

I tried

dependencyOverrides += "org.scala-lang" % "scala-library" % org.scalafmt.Versions.scala % "scalafmt"
// and
libraryDependencies += "org.scala-lang" % "scala-library" % org.scalafmt.Versions.scala % "scalafmt" force()

with no luck. It appears sbt is special handling scala-library. I did find a way to solve this issue however,

libraryDependencies += "org.typelevel" % "scala-library"     % org.scalafmt.Versions.scala   % "scalafmt"

It seems by changing the organisation to org.typelevel then sbt does not special case scala-library anymore. Any better suggestions? cc/ @dwijnand

olafurpg added a commit that referenced this issue Oct 14, 2016

Use typelevel/scala in SBT plugin, fixes #485.
- SBT plugins must build against 2.10
- scalafmt is 2.11 only because scala.meta makes heavy usage of macro
  annotations.
- To solve this problem, the scalafmt SBT plugin classloads itself with
  scala-library 2.11.8 in a hidden configuration (see
    http://www.scala-sbt.org/0.13/docs/Library-Management.html#ivy-configurations)
- This solution is problematic because in 2.12 SBT builds the scala-library:2.11.8
  dependency is evicted in favor of 2.12. Why? It appears SBT special handles
  org.scala-lang:scala-library for some reason.
- This commit changes the SBT plugin to use
org.typelevel:scala-library instead of org.scala-lang:scala-library.

This solution may not work on a project that uses
org.typeleve:scala-library:2.12.  Hopefully everyone will use SBT 1.0
before that happens.

olafurpg added a commit that referenced this issue Oct 14, 2016

Use typelevel/scala in SBT plugin, fixes #485.
- SBT plugins must build against 2.10
- scalafmt is 2.11 only because scala.meta makes heavy usage of macro
  annotations.
- To solve this problem, the scalafmt SBT plugin classloads itself with
  scala-library 2.11.8 in a hidden configuration (see
    http://www.scala-sbt.org/0.13/docs/Library-Management.html#ivy-configurations)
- This solution is problematic because in 2.12 SBT builds the scala-library:2.11.8
  dependency is evicted in favor of 2.12. Why? It appears SBT special handles
  org.scala-lang:scala-library for some reason.
- This commit changes the SBT plugin to use
org.typelevel:scala-library instead of org.scala-lang:scala-library.

This solution may not work on a project that uses
org.typeleve:scala-library:2.12.  Hopefully everyone will use SBT 1.0
before that happens.
@olafurpg

This comment has been minimized.

Show comment
Hide comment
@olafurpg

olafurpg Oct 14, 2016

Member

I opened a pull request to fix this issue. I'll keep it open for a few days to give time for feedback. Please pitch in if you think this solution is unacceptable.

Member

olafurpg commented Oct 14, 2016

I opened a pull request to fix this issue. I'll keep it open for a few days to give time for feedback. Please pitch in if you think this solution is unacceptable.

@dwijnand

This comment has been minimized.

Show comment
Hide comment
@dwijnand

dwijnand Oct 14, 2016

Contributor

I predict this is only an issue from sbt 0.13.12+: accidental fallout from sbt/sbt#2634

Contributor

dwijnand commented Oct 14, 2016

I predict this is only an issue from sbt 0.13.12+: accidental fallout from sbt/sbt#2634

@dwijnand

This comment has been minimized.

Show comment
Hide comment
@dwijnand
Contributor

dwijnand commented Oct 14, 2016

@dwijnand

This comment has been minimized.

Show comment
Hide comment
@dwijnand

dwijnand Oct 14, 2016

Contributor

/cc @eed3si9n too

Contributor

dwijnand commented Oct 14, 2016

/cc @eed3si9n too

@milessabin

This comment has been minimized.

Show comment
Hide comment
@milessabin

milessabin Oct 14, 2016

I'm not clear on what's going on here ... are you trying to have two different versions of scala-library in the same build?

milessabin commented Oct 14, 2016

I'm not clear on what's going on here ... are you trying to have two different versions of scala-library in the same build?

@olafurpg

This comment has been minimized.

Show comment
Hide comment
@olafurpg

olafurpg Oct 15, 2016

Member

@dwijnand I can confirm that 0.13.12 introduced this regression. I am unable to reproduce the issue with 0.13.11. Should I file an issue in sbt/sbt?

@hseeberger A temporary workaround this issue is to use sbt.version=0.13.11

@milessabin The issue appears to be how SBT 0.13.12 resolves org.scala-lang:scala-library between different configurations. The configurations should be isolated but it appears that the 0.13.12 resolver evicts org.scala-lang:scala-library:2.11.8 from the scalafmt configuration in favor of org.scala-lang:scala-libary:2.12 defined in the default/compile configuration.

Ideally, we could cross-build scalafmt to 2.10 but attempts to port scala.meta (an essential dependency) to 2.10 have been unsuccessful, see scalameta/scalameta#295.

Member

olafurpg commented Oct 15, 2016

@dwijnand I can confirm that 0.13.12 introduced this regression. I am unable to reproduce the issue with 0.13.11. Should I file an issue in sbt/sbt?

@hseeberger A temporary workaround this issue is to use sbt.version=0.13.11

@milessabin The issue appears to be how SBT 0.13.12 resolves org.scala-lang:scala-library between different configurations. The configurations should be isolated but it appears that the 0.13.12 resolver evicts org.scala-lang:scala-library:2.11.8 from the scalafmt configuration in favor of org.scala-lang:scala-libary:2.12 defined in the default/compile configuration.

Ideally, we could cross-build scalafmt to 2.10 but attempts to port scala.meta (an essential dependency) to 2.10 have been unsuccessful, see scalameta/scalameta#295.

@dwijnand

This comment has been minimized.

Show comment
Hide comment
@dwijnand

dwijnand Oct 15, 2016

Contributor

Sure you can file an issue. It's a feature interaction.

Speaking with @milessabin we might want to explore:

  • Verifying that the two Scala versions share the same Scala binary version before mediating
  • Doing based on the fact that you're in your own configuration
Contributor

dwijnand commented Oct 15, 2016

Sure you can file an issue. It's a feature interaction.

Speaking with @milessabin we might want to explore:

  • Verifying that the two Scala versions share the same Scala binary version before mediating
  • Doing based on the fact that you're in your own configuration
@hseeberger

This comment has been minimized.

Show comment
Hide comment
@hseeberger

hseeberger Oct 30, 2016

Update: This is still an issue for Scala 2.12.0 (final), scalafmt 0.4.8 and sbt 0.13.13

hseeberger commented Oct 30, 2016

Update: This is still an issue for Scala 2.12.0 (final), scalafmt 0.4.8 and sbt 0.13.13

@olafurpg

This comment has been minimized.

Show comment
Hide comment
@olafurpg

olafurpg Oct 30, 2016

Member

@hseeberger Unfortunately, this bug did not get fixed in sbt 0.13.13, please upvote sbt/sbt#2786 to help prioritze the ticket (hopefully it gets fixed in 0.13.14-M1). The current workaround is still to use sbt 0.13.11.

Member

olafurpg commented Oct 30, 2016

@hseeberger Unfortunately, this bug did not get fixed in sbt 0.13.13, please upvote sbt/sbt#2786 to help prioritze the ticket (hopefully it gets fixed in 0.13.14-M1). The current workaround is still to use sbt 0.13.11.

@lloydmeta lloydmeta referenced this issue Nov 7, 2016

Merged

Feature/2.12 #51

5 of 10 tasks complete
@olafurpg

This comment has been minimized.

Show comment
Hide comment
@olafurpg

olafurpg Nov 9, 2016

Member

A temporary workaround for this issue was posted here sbt/sbt#2786 (comment) Please confirm if this works, it appears quite many users are affected by this issue.

Member

olafurpg commented Nov 9, 2016

A temporary workaround for this issue was posted here sbt/sbt#2786 (comment) Please confirm if this works, it appears quite many users are affected by this issue.

@lloydmeta lloydmeta referenced this issue Nov 10, 2016

Closed

2.12 on-going TODOs #79

5 of 5 tasks complete
@lloydmeta

This comment has been minimized.

Show comment
Hide comment
@lloydmeta

lloydmeta Nov 10, 2016

Contributor

Worked for me.

Contributor

lloydmeta commented Nov 10, 2016

Worked for me.

lloydmeta referenced this issue in lloydmeta/enumeratum Nov 10, 2016

Add a comment to track sbt/sbt#2786 or https://github.com/olafurpg/sc…
…alafmt/issues/485

to know when we can remove the MediatorWorkaround plugin
@olafurpg

This comment has been minimized.

Show comment
Hide comment
@olafurpg

olafurpg Feb 24, 2017

Member

The old sbt plugin has been removed in favor of a thin wrapper around the CLI

Member

olafurpg commented Feb 24, 2017

The old sbt plugin has been removed in favor of a thin wrapper around the CLI

@olafurpg olafurpg closed this Feb 24, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment