Init Scala3 compiler bridge #947
Conversation
The `scala3-compiler-bridge` lives in its own module because it shares almost no sources with the Scala 2 `compiler-bridge`. It is compiled by Scala 3, using `sbt-dotty`, because it depends on the Scala 3 compiler, which is compiled in Scala 3. As soon as `sbt-dotty` is merged into sbt, the `scala3-compiler-bridge` should be able to compiles itself. This first minimal implementation is a direct translation of `sbt-bridge` in `https://github.com/lampepfl/dotty`: - https://github.com/lampepfl/dotty/blob/master/sbt-bridge/src/xsbt/CompilerInterface.java - https://github.com/lampepfl/dotty/blob/master/sbt-bridge/src/xsbt/CachedCompilerImpl.java One minor difference is that it implements the new `xsbti.compile.CompilerInterface2`.
This commit add support for `VirtualFiles`, that has recently been added to Zinc, to the `scala3-compiler-bridge`. It fixes a bug that was reported in sbt/sbt#6007. Also it fixes a bug in the bug in the returned `CompileAnalysis` by populating the `SourceInfos`. The reported warnings were not returned back in `sbt-dotty` after a succefull compilation.
Change the implementation of `DualLoader` to fix the `LinkageError` exception that occurs in the Scala 3 compiler when the `xsbti.*` classes are loaded. The problem is that those classes are loaded by two different `ClassLoaders`: the sbt loader and the scala instance loader. In the fixed implementation, the compiler classes are loaded by the `DualLoader` itself, which extends `URLClassLoader`, so that the subsequent `xsbti.*` classes can be loaded by the underlying `dual` loader which is the sbt loader. See https://github.com/lampepfl/dotty/blob/master/sbt-bridge/src/xsbt/CompilerClassLoader.java for more details.
|
I changed the constructor of Yet, since
I found no usage of |
|
As intended, Mima complains about the hierachy and constructor of Also it complains that That's because I found no usage in Zinc nor sbt nor Bloop and so I decided to remove them. |
|
This
// project/build.properties
sbt.version=1.5.0-SNAPSHOT// project/plugins.sbt
addSbtPlugin("ch.epfl.lamp" % "sbt-dotty" % "0.4.6")// build.sbt
lazy val root = project
.in(file("."))
.settings(
scalaVersion := "3.0.0-M1",
scalaCompilerBridgeBinaryJar := None
) |
|
One thing is that the current implementation does not handle I intend to implement that very soon. |
There was a problem hiding this comment.
Before discussing the details of the PR (I haven't looked closely but it all seems to make sense), I'd like to talk about moving the compiler bridge from the Dotty repo to this repo, because I don't think moving the bridge is a good idea:
- The compiler bridge uses internal Dotty compiler APIs, these APIs are not stable and we reserve the right to change them at anytime even in point releases, which means breaking new Scala 3 releases on existing sbt versions.
- If we do change the API, we run into complicated bootstrapping issues: Scala 3 builds itself using sbt, but if sbt uses an incompatible bridge, we're stuck.
- By contrast, the zinc APIs used in the compiler bridge are pure Java APIs, and an effort has been made recently to make it possible to evolve them without breaking existing users of these APIs (Reluctantly restore compatibility of xsbti.* #829, Make xsbt.CompilerInterface class name configurable #872), the exposed API is also much smaller than the entire Scala 3 compiler jar, so the effort to keep it stable should be much smaller.
So I'm in favor of maintaining the status quo here. I think the ideal solution would be to expand our pure Java APIs in scala3-interfaces to cover everything that the bridge needs to communicate with the compiler, and then it won't matter so much where we put the bridge, but until that happens I would like the bridge to stay next to the compiler.
I also see that your PR rewrote the bridge in Scala: we explicitly rewrote it in Java back in scala/scala3#5596 to avoid complicated bootstrapping issues (you need the compiler to compile the bridge, but to compile the compiler you need the bridge ...), I'd like to keep it that way if possible since it makes everything simpler in my opinion.
|
By the way, it looks like sbt/sbt#6080 already mentioned "Merge sbt-bridge in Zinc." as the proposed implementation. I didn't notice this when I commented on that issue and thus didn't raise any objection to it at the time, sorry for that. |
|
Thank you @smarter for telling this now, it's never too late... :) So basically you want the compiler-bridge to be tied to the exact compiler version: It also means that the Zinc interface must be stable since the scala 3 compiler-bridge does not depend on the Zinc version anymore.
You can still maintain your own bridge in dotty for the sole purpose of bootstrapping. But I guess it is better to have one bridge for all purposes so that the Scala 3 and Zinc API converge.
Right but the old Here the proposed implementation does extend the new
So you expect the binary compatibility between two consecutive versions of Scala 3 to be broken? I propose the following:
|
Just merged: scala/scala3#10498 :)
I'm fine with dropping support for sbt 1.3.
Not now, but eventually with 3.T/4 yes: scala/scala3#10244, having the bridge compiled once and pushed to maven also means sbt doesn't have to compile it with each new compiler version, which means no complicated cache handling (I know mill at least struggled quite a bit to cache the bridge correctly)
Sounds good to me! Though the changes to DualLoader need to be carefully inspected since ClassLoader are always tricky business and the way they are cached in sbt/zinc is also quite intricate (and if classloader caching doesn't work right, it might still work but slow things done as the JIT ends up recompiling the same classes again and again, it helps to run the JVM with |
|
There is still a problem actually which is that implementing So the first thing I'll do is to fix the |
|
I think the goal is to release a 1.5.0 that is fully scala 3 compatible before the scala 3 release. It doesn't necessarily seem worth a lot of effort to get things working with all versions of 1.4.x. |
First step of sbt/sbt#6080.
Minimal
scala3-compiler-bridgeThe
scala3-compiler-bridgelives in its own module because it shares almost no sources with the Scala 2compiler-bridge.It is compiled by Scala 3, using
sbt-dotty, because it depends on the Scala 3 compiler which is compiled in Scala 3.As soon as
sbt-dottyis merged into sbt, thescala3-compiler-bridgewill be able to compiles itself.This first minimal implementation is a direct translation of
sbt-bridgein https://github.com/lampepfl/dotty:One minor difference is that it implements the new
xsbti.compile.CompilerInterface2.Add support for
VirtualFilein Scala 3This commit add support for
VirtualFiles, that has recently been added to Zinc, to thescala3-compiler-bridge.It fixes a bug that was reported in sbt/sbt#6007.
Also it fixes a bug in the returned
CompileAnalysisby populating theSourceInfos.The reported warnings were not returned back in
sbt-dottyafter a succefull compilation.Improve
DualLoaderfor Scala 3Change the implementation of
DualLoaderto fix theLinkageErrorexception that occurs in the Scala 3 compiler when thexsbti.*classes are loaded.The problem is that those classes are loaded by two different
ClassLoaders: the sbt loader and the scala instance loader.In the fixed implementation, the compiler classes are loaded by the
DualLoaderitself, which extendsURLClassLoader, so that the subsequentxsbti.*classes can be loaded by the underlyingdualloader which is the sbt loader.See https://github.com/lampepfl/dotty/blob/master/sbt-bridge/src/xsbt/CompilerClassLoader.java for more details.