Switch projects which use CrossVersion.full/"org.scala-lang" to CrossVersion.patch/scalaOrganization.value

[milessabin]

Please note: the list of projects below is not complete — add any projects you think might benefit from being built and tested with Typelevel Scala

To use Typelevel Scala versions which are not exactly aligned with the corresponding Lightbend Scala version (ie. because they include an additional -bin-patch-nnn suffix in their version) we need to modify project builds which use CrossVersion.full (which includes the suffix) to use CrossVersion.patch (which doesn't) where appropriate (eg. for macro-paradise and other compiler plugins).

Please note that CrossVersion.patch was added in SBT 0.13.13, so projects which aren't already on that version will also need to be upgraded to that version or later.

We must also replace explicit compiler dependencies of the form,

"org.scala-lang" % "scala-reflect" % scalaVersion.value % "provided"
"org.scala-lang" % "scala-compiler" % scalaVersion.value % "provided"

with dependendencies parametrized by scalaOrganization,

scalaOrganization.value % "scala-reflect" % scalaVersion.value % "provided"
scalaOrganization.value % "scala-compiler" % scalaVersion.value % "provided"

The general form of the changes are as shown in the completed/PR'd examples below ...

Please add projects and their status to this list,

• algebra — Change CrossVersion.full to use CrossVersion.patch for Typelevel Scala compatibility. algebra#202
• argonaut — Use CrossVersion.patch and scalaOrganization.value. argonaut-io/argonaut#271
• argonaut-shapeless — Switch from CrossVersion.full to CrossVersion.patch for TLS compatibility alexarchambault/argonaut-shapeless#86
• cats — Switch from CrossVersion.full to CrossVersion.patch for TLS compatibi… cats#1541
• circe — Change CrossVersion.full to use CrossVersion.patch for Typelevel Scala compatibility. circe/circe#580
• config — Cross version lambdista/config#9
• discipline — no changes needed
• dogs — Use CrossVersion.patch. cats-collections#88
• doobie — sbt 0.13.13; crossVersion.full -> .patch tpolecat/doobie#472
• finch — Use scalaOrganization.value. finagle/finch#756
• fs2 — Updated for CrossVersion patch and scalaOrganization fs2#820
• http4s — Added option for compatibility with typelevel scala http4s/http4s#942
• kittens — https://github.com/milessabin/kittens/pull/35
• libisabelle – larsrh/libisabelle@453a5e4
• monix — Switch from CrossVersion.full/"org.scala-lang" to CrossVersion.patch/scalaOrganization.value monix/monix#323
• monocle — update CrossVersion.patch and scalaOrganization.value optics-dev/Monocle#506
• pureconfig — Change use of CrossVersion.full to CrossVersion.patch. pureconfig/pureconfig#158
• refined — Change CrossVersion.full to use CrossVersion.patch for Typelevel Scala compatibility. fthomas/refined#261
• sbt-sensible — https://github.com/fommil/sbt-sensible/pull/8
• sbt-tls-crossproject — Switch to CrossVersion.patch sbt-tls-crossproject#3
• scalacheck — no changes needed
• scalacheck-shapeless — Update CrossVersion.patch. alexarchambault/scalacheck-shapeless#57
• scalaz — Change CrossVersion.full to CrossVersion.patch scalaz/scalaz#1325
• scodec — update sbt.version to 0.13.13; switch to CrossVersion.patch scodec/scodec#96
• seals — Use CrossVersion.patch instead of .full durban/seals#52
• shapeless — milessabin/shapeless@546f045
• shapeless-contrib — Change CrossVersion.full to use CrossVersion.patch for Typelevel Scala compatibility. shapeless-contrib#41
• singleton-ops — switch to CrossVersion.patch; use scalaOrganization.value fthomas/singleton-ops#28
• slickless — no changes needed
• specs2 — etorreborre/specs2@b16c2af
• spire — Use CrossVersion.patch and scalaOrganization.value. spire#638
• spray-json-shapeless — https://github.com/fommil/spray-json-shapeless/pull/34
• tut — "org.scala-lang" -> scalaOrganization.value tpolecat/tut#146
• frameless
• scalatest

[ghost]

Would this also be applicable to sbt plugins? eg: TlsJvmCrossProject and TlsJsCrossProject

[milessabin]

@BennyHill it would only be necessary if you wanted to build an SBT plugin with TLS. But even if that's unlikely it does no harm, so why not?

[ghost]

Added new issue typelevel/sbt-tls-crossproject#3

[tpolecat]

• doobie

[ClydeMachine]

I see shapeless-contrib looks to be using SBT 0.13.13 and isn't in the above list, yet fits the criteria to be changed. Made a PR there all the same in case that fits the bill.

[ClydeMachine]

Grabbed typelevel/algebra as well.

[ghost]

sbt-tls-crossproject done, thanks to @rilut for that 👍

[ClydeMachine]

• algebra

[Atry]

Will https://github.com/milessabin/macro-compat switch to typelevel Scala?

[milessabin]

@Atry There isn't a TLS for 2.10.x so even if there was a need for it (is there?) it isn't possible.

[Atry]

So will shapeless drop the support for 2.10 after switching to TLS?

[rossabaker]

• http4s

[tpolecat]

• tut

[meddulla]

• scalacheck-shapeless

[etorreborre]

• specs2 (3.8.9)

[lambdista]

• config

[sh0hei]

• argonaut-shapeless

[sh0hei]

• pureconfig

[larsrh]

• libisabelle – larsrh/libisabelle@453a5e4

[DanielaSfregola]

Opened PR spray-json-shapeless: see https://github.com/fommil/spray-json-shapeless/pull/34

[sjrd]

So, I received this PR in Scala.js: scala-js/scala-js#2954
I understand you want all compiler plugins to use CrossVersion.patch so that they can be resolved even when scalaVersion is of the form 2.xx.yy-bin-whatever, as if it were 2.xx.yy.

However, for that to be meaningful, there must be a guarantee that 2.xx.yy-bin-whatever (from any organization, mind!) is backwards binary compatible with Lightbend Scala 2.xx.yy. Is there such a guarantee? I have never seen one being spelled out.

If there's no such guarantee, the dependency would resolve but it would be a lie, as it would crash with obscure LinkageErrors.

[dwijnand]

/cc @lrytz who had the same concern in March (scala/scala-dev#252 (comment))

[lrytz]

https://gitter.im/typelevel/scala?at=58e1e0df8fcce56b20c443e3 for an example where this (I think) caused breakage

[sjrd]

That definitely looks like something related, yes.

IMO in general we cannot use CrossVersion.patch, as it is too lenient. In typical "user" builds it might be OK, but in the Scala.js sbt plugin that declares the dependency on the Scala.js compiler plugin, it is too weak. From where I stand, I see two paths forward:

• Either someone publishes the Scala.js compiler plugin (and the Scala.js JUnit compiler plugin) for releases of Typelevel Scala, or
• Something is done in the Typelevel Scala build that ensures it stays backwards binary compatible with Lightbend Scala, and then we could use CrossVersion.patch if scalaOrganization.value is org.typelevel

The second solution is hacky. It wouldn't scale to other versioning schemes from other organizations, etc. The first one is the cleanest, but requires more releases of the Scala.js compiler plugin, and we definitely don't have the CPUs to add Typelevel builds of Scala.js to our CI. Someone else than us could publish Scala.js compiler plugins recompiled for Typelevel, but then we would also need to provide a setting to configure the "scalaJSOrganization" in addition to scalaOrganization.

[ghost]

ref scalacenter/advisoryboard#20 (comment)
Proposal: Prioritize and collate sbt/scala/dbuild/community-build issues for community projects

This is a perfect example, no?

And what about scala.native, https://github.com/typelevel/sbt-tls-crossproject, etc ?

[Atry]

Option 3: Release compiler plugins as source code instead of Java bytecode, like what sbt's compiler interface did.

[sjrd]

But for that you'll need a lot of infrastructure in sbt, which we don't have now.

It also assumes that compiler plugins have the exact same sources for all versions of Scala. This is true for the Scala.js compiler plugin, through significant cleverness that would scare a lot of people away. But it's not true for all compiler plugins, notably macro-paradise.

[Atry]

I think there is a dependsOn method in sbt

dependsOn(ProjectRef(uri("git://github.com/xxx/xxx.git"), "subproject"))

[sjrd]

That assumes it's easy and fast to load the given repo, its build, and compile that project. For significant compiler plugins like Scala.js and paradise, this is far from being true.