Inlining code causes an java.lang.IndexOutOfBoundsException for Scala 2.13

[gergelyfabian]

After upgrading my code from Scala 2.12 to 2.13 I found a bug.

If I inline some code that uses a certain protobuf version I get the following exception from the compiler:

error: java.lang.IndexOutOfBoundsException
	at scala.tools.asm.tree.InsnList.get(InsnList.java:94)
	at scala.tools.nsc.backend.jvm.analysis.BackendUtils$.computeMaxLocalsMaxStack(BackendUtils.scala:728)
	at scala.tools.nsc.backend.jvm.analysis.BackendUtils$.maxLocals(BackendUtils.scala:636)
	at scala.tools.nsc.backend.jvm.opt.Inliner.inlineCallsite(Inliner.scala:856)
	at scala.tools.nsc.backend.jvm.opt.Inliner.$anonfun$runInliner$10(Inliner.scala:352)
	at scala.collection.IterableOnceOps.foreach(IterableOnce.scala:619)
	at scala.collection.IterableOnceOps.foreach$(IterableOnce.scala:617)
	at scala.collection.AbstractIterable.foreach(Iterable.scala:935)
	at scala.collection.IterableOps$WithFilter.foreach(Iterable.scala:905)
	at scala.tools.nsc.backend.jvm.opt.Inliner.runInliner(Inliner.scala:378)
	at scala.tools.nsc.backend.jvm.opt.Inliner.runInlinerAndClosureOptimizer(Inliner.scala:281)
	at scala.tools.nsc.backend.jvm.PostProcessor.runGlobalOptimizations(PostProcessor.scala:156)
	at scala.tools.nsc.backend.jvm.GeneratedClassHandler$GlobalOptimisingGeneratedClassHandler.complete(GeneratedClassHandler.scala:98)
	at scala.tools.nsc.backend.jvm.GenBCode$BCodePhase.$anonfun$run$1(GenBCode.scala:81)
	at scala.tools.nsc.backend.jvm.GenBCode$BCodePhase.run(GenBCode.scala:78)
	at scala.tools.nsc.Global$Run.compileUnitsInternal(Global.scala:1564)
	at scala.tools.nsc.Global$Run.compileUnits(Global.scala:1548)
	at scala.tools.nsc.Global$Run.compileSources(Global.scala:1540)
	at scala.tools.nsc.Global$Run.compile(Global.scala:1674)
	at scala.tools.nsc.Driver.doCompile(Driver.scala:48)
	at scala.tools.nsc.MainClass.doCompile(Main.scala:30)
	at scala.tools.nsc.Driver.process(Driver.scala:68)
	at io.bazel.rulesscala.scalac.ScalacInvoker.invokeCompiler(ScalacInvoker.java:24)
	at io.bazel.rulesscala.scalac.ScalacWorker.compileScalaSources(ScalacWorker.java:272)
	at io.bazel.rulesscala.scalac.ScalacWorker.work(ScalacWorker.java:88)
	at io.bazel.rulesscala.worker.Worker.persistentWorkerMain(Worker.java:96)
	at io.bazel.rulesscala.worker.Worker.workerMain(Worker.java:49)
	at io.bazel.rulesscala.scalac.ScalacWorker.main(ScalacWorker.java:48)
java.lang.IndexOutOfBoundsException
	at scala.tools.asm.tree.InsnList.get(InsnList.java:94)
	at scala.tools.nsc.backend.jvm.analysis.BackendUtils$.computeMaxLocalsMaxStack(BackendUtils.scala:728)
	at scala.tools.nsc.backend.jvm.analysis.BackendUtils$.maxLocals(BackendUtils.scala:636)
	at scala.tools.nsc.backend.jvm.opt.Inliner.inlineCallsite(Inliner.scala:856)
	at scala.tools.nsc.backend.jvm.opt.Inliner.$anonfun$runInliner$10(Inliner.scala:352)
	at scala.collection.IterableOnceOps.foreach(IterableOnce.scala:619)
	at scala.collection.IterableOnceOps.foreach$(IterableOnce.scala:617)
	at scala.collection.AbstractIterable.foreach(Iterable.scala:935)
	at scala.collection.IterableOps$WithFilter.foreach(Iterable.scala:905)
	at scala.tools.nsc.backend.jvm.opt.Inliner.runInliner(Inliner.scala:378)
	at scala.tools.nsc.backend.jvm.opt.Inliner.runInlinerAndClosureOptimizer(Inliner.scala:281)
	at scala.tools.nsc.backend.jvm.PostProcessor.runGlobalOptimizations(PostProcessor.scala:156)
	at scala.tools.nsc.backend.jvm.GeneratedClassHandler$GlobalOptimisingGeneratedClassHandler.complete(GeneratedClassHandler.scala:98)
	at scala.tools.nsc.backend.jvm.GenBCode$BCodePhase.$anonfun$run$1(GenBCode.scala:81)
	at scala.tools.nsc.backend.jvm.GenBCode$BCodePhase.run(GenBCode.scala:78)
	at scala.tools.nsc.Global$Run.compileUnitsInternal(Global.scala:1564)
	at scala.tools.nsc.Global$Run.compileUnits(Global.scala:1548)
	at scala.tools.nsc.Global$Run.compileSources(Global.scala:1540)
	at scala.tools.nsc.Global$Run.compile(Global.scala:1674)
	at scala.tools.nsc.Driver.doCompile(Driver.scala:48)
	at scala.tools.nsc.MainClass.doCompile(Main.scala:30)
	at scala.tools.nsc.Driver.process(Driver.scala:68)
	at io.bazel.rulesscala.scalac.ScalacInvoker.invokeCompiler(ScalacInvoker.java:24)
	at io.bazel.rulesscala.scalac.ScalacWorker.compileScalaSources(ScalacWorker.java:272)
	at io.bazel.rulesscala.scalac.ScalacWorker.work(ScalacWorker.java:88)
	at io.bazel.rulesscala.worker.Worker.persistentWorkerMain(Worker.java:96)
	at io.bazel.rulesscala.worker.Worker.workerMain(Worker.java:49)
	at io.bazel.rulesscala.scalac.ScalacWorker.main(ScalacWorker.java:48)

Scala version: 2.13.16 (could not reproduce with Scala 2.12.20).

Steps to reproduce:

git clone git@github.com:gergelyfabian/bazel-scala-example.git
cd bazel-scala-example
git checkout scala-compiler-bug
bazel build //example-maven:example-maven

Example code:

package mypackage

import com.google.protobuf.CodedOutputStream

object Maven {
  def repro(): Unit = {
    val result = new Array[Byte](0)
    CodedOutputStream.newInstance(result)
  }
}

I use the following scalacopt: -opt-inline-from:**.

Example Bazel config:

scala_library(
    name = "example-maven",
    srcs = glob(["src/main/scala/**/*.scala"]),
    scalacopts = [
        "-opt-inline-from:**",
    ],
    deps = [
        #"@maven//:com_google_protobuf_protobuf_java",
        "@com_google_protobuf//:protobuf_java",
    ],
)

This is a Bazel project, not an sbt one, as I wasn't able to reproduce this issue with sbt.
Related to that, in this example code, if I replace the dependency I use from @com_google_protobuf//:protobuf_java (de-facto built-in Bazel one) to @maven//:com_google_protobuf_protobuf_java (one generated by rules_jvm_external), then I cannot reproduce the bug any more, so it seems to be related to the protobuf version Bazel uses.
That would also explain why I cannot reproduce with sbt, as in sbt I suppose I get the same protobuf-java as with rules_jvm_external.

In any case, I wouldn't expect the Scala compiler to throw such an exception.

[gergelyfabian]

Managed to reproduce with sbt too.

Here is my project:

.
├── build.sbt
├── lib
│   ├── libcore-hjar.jar
│   └── liblite_runtime_only-hjar.jar
├── project
│   └── build.properties
└── src
    └── main
        └── scala
            └── example
                └── Example.scala

build.sbt:

name := "scala-compiler-inline-bug"

version := "0.1"

scalaVersion := "2.13.16"

unmanagedJars in Compile += file("lib/libcore-hjar.jar")

javacOptions := Seq("--release", "21")

scalacOptions += "-target:jvm-21"
scalacOptions += "-opt-inline-from:**"

Example.scala:

package example

import com.google.protobuf.CodedOutputStream

object Example {
  def repro(): Unit = {
    val result = new Array[Byte](0)
    CodedOutputStream.newInstance(result)
  }
}

The two files in lib are added from my example Bazel project with:

# This will fail, but will generate the dependency jars.
( cd ~/opt/bazel-scala-example; bazel build //example-maven:example-maven )
cp ~/opt/bazel-scala-example/bazel-out/k8-fastbuild/bin/external/com_google_protobuf/java/core/libcore-hjar.jar lib/
cp ~/opt/bazel-scala-example/bazel-out/k8-fastbuild/bin/external/com_google_protobuf/java/core/liblite_runtime_only-hjar.jar lib/

Also uploaded these files to Google Drive.

Repro (using sbt only):

git clone git@github.com:gergelyfabian/scala-compiler-inline-bug.git
cd scala-compiler-inline-bug
mkdir -p lib
curl -L -o lib/libcore-hjar.jar 'https://drive.google.com/uc?export=download&id=1NbKjL5vwqoLUb-sbR3vMpKjKBqvF8pio'
curl -L -o lib/liblite_runtime_only-hjar.jar 'https://drive.google.com/uc?export=download&id=12pSp3bTaMZxqkzQP3Ah7SMuv_Co0vxfg'
sbt compile

Produces:

$ sbt compile
[info] welcome to sbt 1.10.2 (Ubuntu Java 21.0.6)
[info] loading global plugins from /home/user/.sbt/1.0/plugins
[info] loading project definition from /home/user/Work/scala-compiler-inline-bug/project
[info] loading settings for project scala-compiler-inline-bug from build.sbt ...
[info] set current project to scala-compiler-inline-bug (in build file:/home/user/Work/scala-compiler-inline-bug/)
[info] Executing in batch mode. For better performance use sbt's shell
[info] compiling 1 Scala source to /home/user/Work/scala-compiler-inline-bug/target/scala-2.13/classes ...
[error] ## Exception when compiling 1 sources to /home/user/Work/scala-compiler-inline-bug/target/scala-2.13/classes
[error] java.lang.IndexOutOfBoundsException
[error] scala.tools.asm.tree.InsnList.get(InsnList.java:94)
[error] scala.tools.nsc.backend.jvm.analysis.BackendUtils$.computeMaxLocalsMaxStack(BackendUtils.scala:728)
[error] scala.tools.nsc.backend.jvm.analysis.BackendUtils$.maxLocals(BackendUtils.scala:636)
[error] scala.tools.nsc.backend.jvm.opt.Inliner.inlineCallsite(Inliner.scala:856)
[error] scala.tools.nsc.backend.jvm.opt.Inliner.$anonfun$runInliner$10(Inliner.scala:352)
[error] scala.collection.IterableOnceOps.foreach(IterableOnce.scala:619)
[error] scala.collection.IterableOnceOps.foreach$(IterableOnce.scala:617)
[error] scala.collection.AbstractIterable.foreach(Iterable.scala:935)
[error] scala.collection.IterableOps$WithFilter.foreach(Iterable.scala:905)
...

[gergelyfabian]

If I change my sbt repro to use Scala 2.12.20, it does not fail compiling:

-scalaVersion := "2.13.16"
+scalaVersion := "2.12.20"

[SethTisue]

I wonder if it's related to #12612 — wdyt @lrytz ?

@gergelyfabian do those JARs exist on the internet, can I grab them with curl? That's the main remaining thing that would make this easier to reproduce.

[gergelyfabian]

I am not sure whether you can grab them from somewhere, but it's easy to generate them. You just need `bazelisk` installed ( https://github.com/bazelbuild/bazelisk), and then run the `bazel build` command as I wrote in the repro steps. That will generate the jars, and I added the path to their location. Btw. I did this on Linux.

…

Message ID: ***@***.***>

[SethTisue]

I am not sure whether you can grab them from somewhere, but it's easy to
generate them. You just need bazelisk installed (
https://github.com/bazelbuild/bazelisk), and then run the bazel build
command as I wrote in the repro steps. That will generate the jars, and I
added the path to their location.

To be honest, this is a level of extra effort that compiler maintainers aren't typically willing to go through. You might get lucky with someone else, but I'm not personally willing.

The ideal actionable compiler bug report doesn't involve any external dependencies whatsoever. Having an external dependency of any kind greatly decreases the likelihood of an investigation occurring.

But if a report must involve an external dependency, that dependency ought to be readily available.

Aren't the JARs on Maven Central somewhere...?

[gergelyfabian]

In the Bazel world everything is generated on the user's side (that's the reason why protobuf is also rebuilt - to make things reproducible). But, I'll soon send a link to the jar files uploaded to an url (from what I built locally).

[SethTisue]

protobuf is also rebuilt

And can we be confident that the resulting JARs aren't somehow corrupt...?

We seek to reduce or eliminate external dependencies in bug reports not only to reduce the effort level for maintainers, but also to narrow down the possible root causes of whatever is going wrong.

[SethTisue]

In the Bazel world everything is generated on the user's side

Is that process intended to produce something that's equivalent to something on Maven Central?

[gergelyfabian]

In the Bazel world everything is generated on the user's side

Is that process intended to produce something that's equivalent to something on Maven Central?

Yes, and if you check the protobuf tar.gz, it includes a Bazel BUILD file that is used to generate the jar file, and is also used to push jars in further targets (by protobuf's CI I suppose) to Maven Central. However, there are different versions of those jars, and I'm not sure it will be exactly the same jar that would end up as a dependency when I use it in my Bazel project.

[gergelyfabian]

protobuf is also rebuilt

And can we be confident that the resulting JARs aren't somehow corrupt...?

We seek to reduce or eliminate external dependencies in bug reports not only to reduce the effort level for maintainers, but also to narrow down the possible root causes of whatever is going wrong.

I can never be sure, as this is a bug that happens in the interaction of Bazel with Scala (and there is rules_scala in the middle too). Obviously there can be an issue on Bazel or rules_scala side, but that would mean that either Bazel builds a wrong jar (for protobuf) or that rules_scala provides a wrong jar to the scalac compiler. I'd doubt that though as for Scala 2.12 there is no issue, only with Scala 2.13 the problem appears. And the same thing is with my sbt repro.

[gergelyfabian]

Uploaded the two jars:

• libcore-hjar.jar: https://drive.google.com/file/d/1NbKjL5vwqoLUb-sbR3vMpKjKBqvF8pio/view?usp=drive_link
• liblite_runtime_only-hjar.jar: https://drive.google.com/file/d/12pSp3bTaMZxqkzQP3Ah7SMuv_Co0vxfg/view?usp=drive_link

[gergelyfabian]

With curl:

curl -L -o lib/libcore-hjar.jar 'https://drive.google.com/uc?export=download&id=1NbKjL5vwqoLUb-sbR3vMpKjKBqvF8pio'
curl -L -o lib/liblite_runtime_only-hjar.jar 'https://drive.google.com/uc?export=download&id=12pSp3bTaMZxqkzQP3Ah7SMuv_Co0vxfg'

[gergelyfabian]

Here is the updated repro:

git clone git@github.com:gergelyfabian/scala-compiler-inline-bug.git
cd scala-compiler-inline-bug
mkdir -p lib
curl -L -o lib/libcore-hjar.jar 'https://drive.google.com/uc?export=download&id=1NbKjL5vwqoLUb-sbR3vMpKjKBqvF8pio'
curl -L -o lib/liblite_runtime_only-hjar.jar 'https://drive.google.com/uc?export=download&id=12pSp3bTaMZxqkzQP3Ah7SMuv_Co0vxfg'
sbt compile

[som-snytt]

There was another issue with ASM where @lrytz asked them to take a fix; I don't have a link. That's to say, it's not inconceivable that a strange build product interacts badly with ASM facilities.