Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

scalac's flags -opt-trace and -opt-log-inline produce no output #11746

Open
ryanberckmans opened this issue Sep 21, 2019 · 11 comments

Comments

@ryanberckmans
Copy link

commented Sep 21, 2019

On OSX with Scala 2.13.1, sbt 1.3.2

java version "1.8.0_25"
Java(TM) SE Runtime Environment (build 1.8.0_25-b17)
Java HotSpot(TM) 64-Bit Server VM (build 25.25-b02, mixed mode)

How do I see what the scalac optimizer/inliner is doing? I have a pretty vanilla sbt project using Scala 2.13.1. I've tried to turn on as many optimizer flags as possible.

sbt / scalac produces no optimizer output of any kind. The flags are

scalacOptions ++= Seq(
  "-opt:_", // enable all optimizations; the "_" means all optimizations
  "-opt-inline-from:**", // enable inlining for all classes
  "-opt-warnings:_", // enable optimizer warnings: `_' for all, `-opt-warnings:help' to list choices
  "-Yopt-trace", "_", // log the optimizer progress; "_" means all methods
  "-Yopt-log-inline", "_", // log the inliner's work; "_" means all methods
)

problem

Optimizer logging flags produce no output. Build succeeds silently.

expectation

Optimizer logging of some kind.

@ryanberckmans ryanberckmans changed the title -opt-trace and -opt-log-inline produce no output with Scala 2.13.1 and sbt 1.3.2 scalac's flags -opt-trace and -opt-log-inline produce no output with Scala 2.13.1 and sbt 1.3.2 Sep 21, 2019
@ryanberckmans ryanberckmans changed the title scalac's flags -opt-trace and -opt-log-inline produce no output with Scala 2.13.1 and sbt 1.3.2 scalac's flags -opt-trace and -opt-log-inline produce no output Sep 21, 2019
@som-snytt

This comment has been minimized.

Copy link

commented Sep 22, 2019

I don't think -opt:_ means what you think.

$ scala -opt:inline -opt-inline-from:** -Vinline _
Welcome to Scala 2.13.0 (OpenJDK 64-Bit Server VM, Java 11.0.3).
Type in expressions for evaluation. Or try :help.

scala> locally(42)
Inlining into $line3/$read$$iw$$iw$.<clinit>
 inlined scala/Predef$.locally (the callee is annotated `@inline`). Before: 16 ins, after: 28 ins.
 inlined scala/runtime/BoxesRunTime.boxToInteger (the callee is a forwarder method with boxing adaptation). Before: 28 ins, after: 36 ins.
Inlining into $line3/$eval$.$print$lzycompute: inlined scala/runtime/BoxesRunTime.boxToInteger (the callee is a forwarder method with boxing adaptation). Before: 56 ins, after: 64 ins.
res0: Int = 42

Edit: I'm sure there is a perfectly reasonable explanation:

$ scala -opt:_
Welcome to Scala 2.13.0 (OpenJDK 64-Bit Server VM, Java 11.0.3).
Type in expressions for evaluation. Or try :help.

scala> $intp.settings.opt.contains($intp.settings.optChoices.inline)
res0: Boolean = true

scala> $intp.settings.optInlinerEnabled
res1: Boolean = false

Edit: because wildcard turns on everything including l:none which disables everything.

scala> $intp.settings.opt.contains($intp.settings.optChoices.lNone)
res0: Boolean = true
@som-snytt

This comment has been minimized.

Copy link

commented Sep 22, 2019

A workaround is to turn off the kill switch:

$ scala -opt:-l:none,_ -opt-warnings:-none,_
Welcome to Scala 2.13.0 (OpenJDK 64-Bit Server VM, Java 11.0.3).
Type in expressions for evaluation. Or try :help.

scala> $intp.settings.opt.contains($intp.settings.optChoices.lNone)
res0: Boolean = false

scala> $intp.settings.optInlinerEnabled
res1: Boolean = true
@ryanberckmans

This comment has been minimized.

Copy link
Author

commented Sep 23, 2019

Thanks, @som-snytt. My initial reference was https://www.lightbend.com/blog/scala-inliner-optimizer. I've been playing with the optimizer/inliner for most of the day. Here's what I learned

[warn] foo.scala:2059:70: failed to determine if isEmpty should be inlined:
[warn] The method isEmpty()Z could not be found in the class java/lang/Object or any of its parents.
@ryanberckmans

This comment has been minimized.

Copy link
Author

commented Sep 23, 2019

I was able to compile with

scalacOptions ++= Seq(
  // The optimizer can sometimes print large numbers
  // of warnings and we'd like to see all of them.
  "-Xmaxwarns", "1000", 

  // Enable all types of optimizations.
  "-opt:-l:none,_",

  // Enable inlining for all classes.
  "-opt-inline-from:**", 

  // Show all optimizer warnings.
  // "-opt-warnings:-none,_",

  // Trace the optimizer progress for methods; `_` to print all,
  // prefix match to select. -Vopt replaced -Yopt-trace
  // "-Vopt", "_", 

  // Print a summary of inliner activity; `_` to print all,
  // prefix match to select. -Vinline replaced -Yopt-log-inline
  // "-Vinline", "_",
)
@som-snytt

This comment has been minimized.

Copy link

commented Sep 23, 2019

Thanks for the update.

There is WIP doc for options at https://docs.scala-lang.org/overviews/compiler-options/index.html

I'll see if there is a quick way to fix the -opt:_ gotcha. I don't know about the subtleties of backend reporting, but I would guess that they are subtle.

@lrytz

This comment has been minimized.

Copy link
Member

commented Sep 23, 2019

Thanks @ryanberckmans raising these issues!

-opt:_ should work, it just never crossed my mind, I hope it's an easy fix. So far we always used and recommended -opt:l:inline to enable all optimizations. I also realize now the difference between -opt:inline and -opt:l:inline could be confusing, we should probably rename l:inline.

-opt-warnings:_ is used a lot in our tests, I need to take another look if it really doesn't work as expected.

I'll also adjust the blog post to mention the new -V flags.

-opt-warnings:-none,_ for me seems to trigger compile errors that fail the build, instead of warnings. Is this a bug? [1]

They shold be warnings. Do you have -Werror or -Xfatal-warnings enabled?

with the inliner enabled, use of @inline when the method can't be inlined seems to cause an optimizer error and not warning, even if -opt-warnings is unset

The default -opt-warnings is at-inline-failed, as shown by scalac -opt-warnings:help. You can specify -opt-warnings:none. They should be warnings though (see my question above).

the options for -Yopt-inline-heuristics seem to be omitted from scaladoc

I don't fully understand, but note that -Y flags are genarlly considered internal / experimental.

[error] foo.scala:2059:70: failed to determine if isEmpty should be inlined:
[error] The method isEmpty()Z could not be found in the class java/lang/Object or any of its parents.

This one I would be interested to take a look. Can you share your project?

@ryanberckmans

This comment has been minimized.

Copy link
Author

commented Sep 24, 2019

@lrytz ugh, I did in fact have -Xfatal-warnings set. I turned it on years ago for this project and forgot about it. Noticing this earlier would have saved me hours. Thanks. 😱

I was able to drop !scala.runtime.Statics from -opt-inline-from as these were just inliner warnings and not errors. Compilation is successful with -opt-inline-from:**.

Tomorrow I will update my posts in this thread to reflect this info so they are correct for any future readers. I will also try to share a snippet related to the failed to determine if isEmpty should be inlined inliner warning, but cannot share the whole project.

@ryanberckmans

This comment has been minimized.

Copy link
Author

commented Sep 25, 2019

Let me try to recap the open issues in this thread

  1. "-opt:_ should work". Potentially we might rename this issue to "make -opt:_ work". But I will cede any stewardship from here. Thanks for the help!

  2. I can confirm that all inliner warnings I was seeing before don't actually fail my build (now that -Xfatal-warnings is unset 🤦). They are just warnings.

  3. for this inliner warning mentioned above

[error] foo.scala:2059:70: failed to determine if isEmpty should be inlined:
[error] The method isEmpty()Z could not be found in the class java/lang/Object or any 

@lrytz most of these warnings are related to accessing methods on array elements or match/case statements. Here are a couple

[error] foo.scala:243:24: failed to determine if value should be inlined:
[error] The method value()Ljava/lang/Object; could not be found in the class java/lang/Object or any of its parents.
[error]           case Success(false) => {
[error]                        ^

[error] foo.scala:836:27: failed to determine if mod should be inlined:
[error] The method mod()Lfoo/stats/Modifier; could not be found in the class java/lang/Object or any of its parents.
[error]             a.statMods(i).mod match {
[error]                           ^

Here's one with code

private def get[A](getA: String => A, path: String)(implicit log: LoggingAdapter, config: Config): Option[A] = Try(getA(path)) match {
  case Success(a) => Some(a)
  case Failure(e: ConfigException.Missing) => None // many settings are optional and we don't want to log these
  case Failure(e) => { // line 93 from warn below
    log.warning(s"config failed to get $path: $e")
    None
  }
}

And the corresponding inliner warning

// See "line 93" comment in snippet above
[warn] foo.scala:93:18: failed to determine if exception should be inlined:
[warn] The method exception()Ljava/lang/Throwable; could not be found in the class java/lang/Object or any of its parents.
[warn]     case Failure(e) => {
[warn]                  ^
@lrytz lrytz self-assigned this Sep 27, 2019
@lrytz lrytz added the optimizer label Sep 27, 2019
@lrytz

This comment has been minimized.

Copy link
Member

commented Sep 27, 2019

Self-contained example:

trait Try
object Try {
  def apply(s: String): Try = Success(s)
}
case class Failure(e: Throwable) extends Try

class C {
  private def get(a: String): Unit = Try(a) match {
    case Failure(e: Exception) => 
    case Failure(e) => println(e.toString)
  }
}
$> scalac Test.scala -opt:l:inline -opt-inline-from:'**' -opt-warnings:_
Test.scala:10: warning: failed to determine if e should be inlined:
The method e()Ljava/lang/Throwable; could not be found in the class java/lang/Object or any of its parents.
    case Failure(e) => println(e.toString)
                 ^
one warning found
@som-snytt

This comment has been minimized.

Copy link

commented Sep 27, 2019

Surprisingly difficult to pass information from ScalaSettings to MutableSettings.

Simplest was to let the choice enum define what wildcard expands to.

@som-snytt

This comment has been minimized.

Copy link

commented Oct 2, 2019

The warning setting -opt-warnings:none,at-inline-failed-summary doesn't negate in the same way? I added the self-contained test with -opt-warnings:none,_ to show that warning is still emitted.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants
You can’t perform that action at this time.