Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP: SD-228 eliminate static super accessors in traits #5415

Closed
wants to merge 6 commits into from

Conversation

lrytz
Copy link
Member

@lrytz lrytz commented Sep 22, 2016

WIP, don't merge: the first commit updates starr to a version on scala-release-temp that corresponds to the tip of this PR. This way I can find out if all commits in this PR pass CI validation.

The temporary starr is build from this branch: https://github.com/lrytz/scala/tree/traitSuperAccessorsTmp
It's the same as this PR, but without the starr update in the first commit. The bootstrap job that built it passed all tests (https://scala-ci.typesafe.com/view/scala-2.12.x/job/scala-2.12.x-integrate-bootstrap/593/)

For submission, the right procedure is probably to keep the current starr, have CI fail on the commits, and then re-starr right after merging it.

@scala-jenkins scala-jenkins added this to the 2.12.0-RC2 milestone Sep 22, 2016
@sjrd
Copy link
Member

sjrd commented Sep 22, 2016

FYI, I suspect this might break Scala.js, more specifically the commit "Make $init$ methods static during erasure instead of code gen". It might be good to launch a community build when this PR approaches merging. Or ping me to perform a local test on SJS.

@lrytz lrytz force-pushed the traitSuperAccessors branch 3 times, most recently from 445be20 to 51298b5 Compare September 22, 2016 17:48
For details see SD-228.

No longer generate static accessor methods for every non-private trait
method. Super calls are implemented using invokespecial. If necessary,
the target interface is added as a direct parent to the current class.
All callsites to $init$ methods are generated during erasure. By having
the method already static at this point, we can avoid the non-local
transform and generate T.$init$(this) from the start, instead of
this.$init$().

This also allows eliminating some now unneeded code in the backend.
Now that we re-introduced the logic to add additional parent interfaces
in the backend to support arbitrary super calls to indirect parent
interfaces, we can do the same if the interface is java-defined.

Before, when we were using static accessors to emit super calls,
indirect java interfaces were excluded.
When adding an interface to a class during code gen, record that fact
in the corresponding BType. This ensures that ClassBTypes constructed
from symbol and classfile are identical.

This is basically the same commit as 3fca034 (which was later reverted).
This reverts 91b066a. Interfaces required for an invokespecial call
are added as direct parent to the class, no matter if they are defined
in Java or Scala.
@lrytz
Copy link
Member Author

lrytz commented Sep 23, 2016

From the community build:

[scala-js] [error] error: java.lang.AssertionError: assertion failed:
[scala-js] [error]   Trying to access the this of another class: tree.symbol = trait LowPrioAnyImplicits, class symbol = object Any compilation unit:Any.scala
[scala-js] [error]      while compiling: /home/jenkins/workspace/scala-2.12.x-integrate-community-build/target-0.9.5/project-builds/scala-js-92f0738043e86c3eecf15c4e0c8376f84b56faf5/library/src/main/scala/scala/scalajs/js/Any.scala

That looks like an invalid AST from the Scala compiler (https://github.com/scala-js/scala-js/blob/master/compiler/src/main/scala/org/scalajs/core/compiler/GenJSCode.scala#L1625-L1635).

I'll try to reproduce..

The rest of the community build is running here: https://scala-ci.typesafe.com/view/scala-2.12.x/job/scala-2.12.x-integrate-community-build/620/console

@lrytz
Copy link
Member Author

lrytz commented Sep 23, 2016

@sjrd here's a minimized example - it doesn't take much:

trait T { val x = 0 } // has an $init$ method
class C extends T   // calls T.$init$(C.this)

The $init$ method is static. The core issue is that there's no sensible AST corresponding to a reference to a static method:

  • Ident(staticMethodSym) doesn't make sense: the method belongs to some class
  • Select(CompanionModule, staticMethodSym) also doesn't make sense: the method is not in the module class
  • Select(Class, staticMethodSym) also doesn't make sense, Class is a type, not a term

The current result of gen.mkAttributedRef(staticMethodSym) is Select(This(ownerClass), methodSym), which is weird too, but it's a way to get from a class symbol to a term tree.

The reason things don't crash in the JVM backend (we have the same assertion in place that crashed in SJS) is that for static calls, we simply skip code gen for the qualifier: https://github.com/scala/scala/blob/2.12.x/src/compiler/scala/tools/nsc/backend/jvm/BCodeBodyBuilder.scala#L669

This is not the case in the SJS code gen, and genExpr(receiver) leads to the crash: https://github.com/scala-js/scala-js/blob/master/compiler/src/main/scala/org/scalajs/core/compiler/GenJSCode.scala#L2305

What do you think is the best way forward? I guess so far you don't have support for static methods in SJS?

@lrytz
Copy link
Member Author

lrytz commented Sep 23, 2016

@sjrd
Copy link
Member

sjrd commented Sep 23, 2016

@lrytz Thanks for the detailed explanation. I guess the best way forward is for SJS to do a similar exception to avoid genExpr(receiver). In fact we should somehow end up with the same treatment as for impl class method calls: https://github.com/scala-js/scala-js/blob/master/compiler/src/main/scala/org/scalajs/core/compiler/GenJSCode.scala#L2298-L2299

The SJS IR has static methods (we've been using them for impl class methods for ages), so that is not a problem.

I'll work on this as soon as the camera ready for my Scala'16 paper is submitted (the deadline is tomorrow noon-ish).

@lrytz
Copy link
Member Author

lrytz commented Sep 23, 2016

@sjrd thanks! No rush, and if it looks non-trivial, please wait until we reach a decision whether this PR gets merged or not - it's still open (scala/scala-dev#228).

@sjrd
Copy link
Member

sjrd commented Sep 24, 2016

@lrytz What resolvers do I need to get nightlies of scala-xml? The following was not enough:

> set resolvers in Global += "pr" at "https://scala-ci.typesafe.com/artifactory/scala-pr-validation-snapshots/"
> ++2.12.0-31505fe-SNAPSHOT

gave me

[warn]  ::::::::::::::::::::::::::::::::::::::::::::::
[warn]  ::          UNRESOLVED DEPENDENCIES         ::
[warn]  ::::::::::::::::::::::::::::::::::::::::::::::
[warn]  :: org.scala-lang.modules#scala-xml_2.12.0-744b296-nightly;1.0.5: not found
[warn]  ::::::::::::::::::::::::::::::::::::::::::::::
[warn]
[warn]  Note: Unresolved dependencies path:
[warn]          org.scala-lang.modules:scala-xml_2.12.0-744b296-nightly:1.0.5
[warn]            +- org.scala-lang:scala-compiler:2.12.0-31505fe-SNAPSHOT (C:\Users\Sepi\Documents\Projets\scalajs\project\Build.scala#L515)
[warn]            +- org.scala-js:scalajs-compiler_2.12.0-31505fe-SNAPSHOT:0.6.13-SNAPSHOT ()

@SethTisue
Copy link
Member

SethTisue commented Sep 24, 2016

What resolvers do I need to get nightlies of scala-xml

@sjrd scripts/jobs/validate/publish-core doesn't build scala-xml. so I don't think the artifact you need exists.

I started this Jenkins run https://scala-ci.typesafe.com/view/scala-2.12.0/job/scala-2.12.0-release-main/82/ which should publish what you need.

@SethTisue
Copy link
Member

SethTisue commented Sep 24, 2016

alternatively, you can clone the scala-xml, repo add one more resolver set resolvers in Global += "nightly" at "https://scala-ci.typesafe.com/artifactory/scala-release-temp/", and ++2.12.0-31505fe-SNAPSHOT publishLocal should work (it worked for me, but I didn't try to use the resulting jar from Scala.js)

@sjrd
Copy link
Member

sjrd commented Sep 24, 2016

Thanks @SethTisue, I manage to get it from the artifacts published by the build you mentioned.

I think the necessary changes are those in scala-js/scala-js@master...sjrd:static-trait-init-rc2
However, I get a very strange error when compiling the helloworld with this branch (> helloworld/compile). After compiling the library and eventually arriving at the helloworld, I get:

[error] error while loading package, class file 'C:\Users\Sepi\Documents\Projets\scalajs\library\target\scala-2.12.0-315
05fe-SNAPSHOT\scalajs-library_2.12.0-31505fe-SNAPSHOT-0.6.13-SNAPSHOT.jar(scala/scalajs/js/package.class)' is broken
[error] (class java.lang.NullPointerException/null)
[error] error while loading JSApp, class file 'C:\Users\Sepi\Documents\Projets\scalajs\library\target\scala-2.12.0-31505
fe-SNAPSHOT\scalajs-library_2.12.0-31505fe-SNAPSHOT-0.6.13-SNAPSHOT.jar(scala/scalajs/js/JSApp.class)' is broken
[error] (class java.lang.NullPointerException/null)
[error] C:\Users\Sepi\Documents\Projets\scalajs\examples\helloworld\HelloWorld.scala:14: illegal inheritance;
[error]  self-type helloworld.HelloWorld.type does not conform to scala.scalajs.js.JSApp's selftype scala.scalajs.js.JSA
pp
[error] object HelloWorld extends js.JSApp {
[error]                              ^
[error] three errors found
[error] (helloworld/compile:compileIncremental) Compilation failed

Note that this is about class files being broken.

If I revert the change at declaration site of the $init$ method, i.e., at scala-js/scala-js@master...sjrd:static-trait-init-rc2#diff-c9a195a739d51194e79f0655bc93bd61R1322 (not at call site), and I rebuild everything (so after a full clean), then I can successfully compile the hello world (although not link it, obviously, since the call site and definition site don't match at the IR level).

If I revert the change when compiling the library (library/package), and then enable the change when compiling the hello world (helloworld/compile), compilation succeeds. So this seems to indicate that the issue happens when building the class files for the library, not when loading them.

Only accessing sym.isStaticMember without using its value in the test is not sufficient to trigger the compile error, so it doesn't seem to be an issue caused by forcing. Yet somehow the presence of this test causes the class files to be broken or not.

I'm a bit lost right now. I think I'll let it rest until Monday, unless you guys have a hint for me.

FTR, I'm on Windows atm.

@lrytz
Copy link
Member Author

lrytz commented Sep 26, 2016

@sjrd I did clean and helloworld/compile, and it succeeded for me. here's how i updated the scala version: https://github.com/scala-js/scala-js/compare/master...lrytz:static-trait-init-rc2?expand=1

I used the same scala version that's used in this PR as starr, it's published (together with modules) on scala-release-temp.

@lrytz
Copy link
Member Author

lrytz commented Sep 26, 2016

At least it looks to me like it succeeded: some warnings are reported in sbt as [error], but at the end there's a [success]:

> helloworld/compile
...
[info] Compiling 33 Scala sources to /Users/luc/scala/scala-js/compiler/target/scala-2.12.0-744b296-nightly/classes...
...
[info] Resolving org.scala-js#scalajs-compiler_2.12.0-744b296-nightly;0.6.13-SNAPSHOT ...
...
[warn]  ::::::::::::::::::::::::::::::::::::::::::::::
[warn]  ::          UNRESOLVED DEPENDENCIES         ::
[warn]  ::::::::::::::::::::::::::::::::::::::::::::::
[warn]  :: org.scala-js#scalajs-compiler_2.12.0-744b296-nightly;0.6.13-SNAPSHOT: not found
[warn]  ::::::::::::::::::::::::::::::::::::::::::::::
[info] Unpacking Scala library sources to /Users/luc/scala/scala-js/scalalib/target/scalaSources/2.12.0-744b296-nightly...
...
[info] Packaging /Users/luc/scala/scala-js/compiler/target/scala-2.12.0-744b296-nightly/scalajs-compiler_2.12.0-744b296-nightly-0.6.13-SNAPSHOT.jar ...
[info] Done packaging.
[info] Compiling 102 Scala sources to /Users/luc/scala/scala-js/library/target/scala-2.12.0-744b296-nightly/classes...
[info] Compiling 4 Scala sources to /Users/luc/scala/scala-js/library-aux/target/scala-2.12.0-744b296-nightly/classes...
[info] Compiling 39 Scala sources to /Users/luc/scala/scala-js/javalanglib/target/scala-2.12.0-744b296-nightly/classes...
[info] Compiling 165 Scala sources to /Users/luc/scala/scala-js/javalib/target/scala-2.12.0-744b296-nightly/classes...
[info] Compiling 544 Scala sources to /Users/luc/scala/scala-js/scalalib/target/scala-2.12.0-744b296-nightly/classes...
[error] warning: 'icode' specifies no phase
[error] warning: 'icode' specifies no phase
[error] warning: 'icode' specifies no phase
[error] warning: 'icode' specifies no phase
[error] warning: there were 38 deprecation warnings (since 2.10.0)
[error] warning: there were 20 deprecation warnings (since 2.11.0)
[error] warning: there were two deprecation warnings (since 2.12)
[error] warning: there were 43 deprecation warnings (since 2.12.0)
[error] warning: there were 103 deprecation warnings in total; re-run with -deprecation for details
[error] warning: there was one feature warning; re-run with -feature for details
[error] 6 warnings found
[info] Packaging /Users/luc/scala/scala-js/library/target/scala-2.12.0-744b296-nightly/scalajs-library_2.12.0-744b296-nightly-0.6.13-SNAPSHOT.jar ...
[info] Done packaging.
[info] Compiling 1 Scala source to /Users/luc/scala/scala-js/examples/helloworld/target/scala-2.12.0-744b296-nightly/classes...
[success] Total time: 116 s, completed Sep 26, 2016 12:52:53 PM

@sjrd
Copy link
Member

sjrd commented Sep 26, 2016

Indeed, with your suggested settings to use the nightly version, I can compile and run the helloworld, as well as the test suite.

At least it looks to me like it succeeded: some warnings are reported in sbt as [error], but at the end there's a [success]:

Those [error]s are expected. They're not [warn]s because the standard library is compiled with a forked scalac (instead of through sbt's incremental compiler) so we only have the stdout/stderr distinction.

@sjrd
Copy link
Member

sjrd commented Sep 26, 2016

Here is a PR for Scala.js: scala-js/scala-js#2608 so that our full CI runs on it.
Pending acceptance of the present PR in scala/scala

@lrytz
Copy link
Member Author

lrytz commented Sep 26, 2016

Thanks for checking, @sjrd! Here's a re-run of the community built with scala-js pinned to your branch.

https://scala-ci.typesafe.com/view/scala-2.12.x/job/scala-2.12.x-integrate-community-build/663/console

@adriaanm adriaanm added the WIP label Sep 26, 2016
@lrytz
Copy link
Member Author

lrytz commented Sep 28, 2016

we decided that we're not going to merge this, see scala/scala-dev#228 (comment)

@lrytz lrytz closed this Sep 28, 2016
lrytz added a commit to lrytz/scala that referenced this pull request Sep 28, 2016
Recovered and adapted some test cases for super calls from scala#5415
lrytz added a commit to lrytz/scala that referenced this pull request Sep 28, 2016
Recovered and adapted some test cases for super calls from scala#5415
lrytz added a commit to lrytz/scala that referenced this pull request Sep 30, 2016
Recovered and adapted some test cases for super calls from scala#5415
lrytz added a commit to lrytz/scala that referenced this pull request Sep 30, 2016
Recovered and adapted some test cases for super calls from scala#5415
lrytz added a commit to lrytz/scala that referenced this pull request Sep 30, 2016
Recovered and adapted some test cases for super calls from scala#5415
lrytz added a commit to lrytz/scala that referenced this pull request Sep 30, 2016
Recovered and adapted some test cases for super calls from scala#5415
lrytz added a commit to lrytz/scala that referenced this pull request Sep 30, 2016
Recovered and adapted some test cases for super calls from scala#5415
lrytz added a commit to lrytz/scala that referenced this pull request Sep 30, 2016
Recovered and adapted some test cases for super calls from scala#5415
lrytz added a commit to lrytz/scala that referenced this pull request Sep 30, 2016
Recovered and adapted some test cases for super calls from scala#5415
adriaanm pushed a commit to lrytz/scala that referenced this pull request Sep 30, 2016
Recovered and adapted some test cases for super calls from scala#5415
@SethTisue SethTisue removed this from the 2.12.0-RC2 milestone Oct 14, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
5 participants