REPL - invoke pprint reflectively #24119

bishabosha · 2025-10-01T15:54:34Z

reset the print function each time the classloader resets, if cant load reflectively, then fallback to calling directly.

before:

scala -S 3.nightly
Downloading Scala 3.8.0-RC1-bin-20251003-172687a-NIGHTLY compiler
Downloading Scala 3.8.0-RC1-bin-20251003-172687a-NIGHTLY bridge
Welcome to Scala 3.8.0-RC1-bin-20251003-172687a-NIGHTLY-git-172687a (17.0.14, Java OpenJDK 64-Bit Server VM).
Type in expressions for evaluation. Or try :help.
                                                                                                                                 
scala> (1, 2L, 3.0F, "hello", 'a')
val res0: (Int, Long, Float, String, Char) = (1,2,3.0,hello,a)
                                                                                                                                 
scala>

after:

Welcome to Scala 3.8.0-RC1-bin-SNAPSHOT-git-ad52fbe (17.0.14, Java OpenJDK 64-Bit Server VM).
Type in expressions for evaluation. Or try :help.
                                                                                                                                 
scala> (1, 2L, 3.0F, "hello", 'a')
val res0: (Int, Long, Float, String, Char) = (1, 2L, 3.0F, "hello", 'a')
                                                                                                                                 
scala>

bishabosha · 2025-10-01T15:57:12Z

i think breakage in the interfaces would become clear as repl tests would fail.

But to isolate the classpath issue in a test, perhaps sbt scripted test is appropriate?

bishabosha · 2025-10-01T16:12:13Z

another way i guess is compiling a header script with a wrapper print method and invoking that reflectively,

or perhaps there is some way to manipulate the classloaders to avoid reflection?

He-Pin · 2025-10-01T16:55:37Z

compiler/src/dotty/tools/repl/Rendering.scala

+        classOf[Int],     // initialOffset
+        classOf[Boolean], // escape Unicode
+        classOf[Boolean], // show field names
+      )


with methodHandle？

this isn't well suited because method handle is only optimised if its a static final field, but this is based on a dynamic class loader

lihaoyi · 2025-10-01T16:56:04Z

compiler/src/dotty/tools/repl/Rendering.scala

+      BlackWhite_apply = pprintCls.getMethod("apply",
+        classOf[Any],     // value
+        classOf[Int],     // width
+        classOf[Int],     // height
+        classOf[Int],     // indentation
+        classOf[Int],     // initialOffset
+        classOf[Boolean], // escape Unicode
+        classOf[Boolean], // show field names
+      )
+      val fansiStrCls = Class.forName("dotty.shaded.fansi.Str", false, cl)
+      FansiStr_plainText = fansiStrCls.getMethod("plainText")


Could this be more cleanly written using reflective structural types?

BlackWhite .asInstanceOf[{ def apply( value: Any, width: Int, height: Int, indentation: Int, initialOffset: Int, escapeUnicode: Boolean, showFieldNames: Boolean): { def plainText: String } }] .apply(value, width, height, 2, initialOffset, false, true) .plainText

although this still doesn't statically protect against the actual api changing

Yeah it would be just as safe/unsafe as it is now, just a bit easier to read

bishabosha · 2025-10-02T08:32:18Z

fails with java.lang.ClassNotFoundException: dotty.shaded.pprint.PPrinter$BlackWhite$ in test-scala3-compiler-bootstrapped and test-scala3-compiler-nonbootstrapped for ReplCompilerTests

2025-10-01T17:11:28.8699192Z [info] Test run dotty.tools.dotc.core.tasty.TastyHeaderUnpicklerTest finished: 0 failed, 0 ignored, 20 total, 0.014s
2025-10-01T17:11:28.8713906Z [info] Test run dotty.tools.repl.ReplCompilerTests started
2025-10-01T17:11:28.8719700Z [info] Test dotty.tools.repl.ReplCompilerTests.testSingletonPrint started
2025-10-01T17:11:29.0459423Z [error] Test dotty.tools.repl.ReplCompilerTests.testSingletonPrint failed: org.junit.ComparisonFailure: expected:<[val a: String = "hello"
2025-10-01T17:11:29.0487191Z [error] val x: a.type = "hello"]> but was:<[java.lang.ClassNotFoundException: dotty.shaded.pprint.PPrinter$BlackWhite$
2025-10-01T17:11:29.0488430Z [error]   at dotty.tools.repl.AbstractFileClassLoader.findClass$$anonfun$1(AbstractFileClassLoader.scala:49)
2025-10-01T17:11:29.0489551Z [error]   at scala.runtime.function.JProcedure1.apply(JProcedure1.java:15)
2025-10-01T17:11:29.0490372Z [error]   at scala.runtime.function.JProcedure1.apply(JProcedure1.java:10)
2025-10-01T17:11:29.0491175Z [error]   at scala.collection.immutable.List.foreach(List.scala:327)
2025-10-01T17:11:29.0492109Z [error]   at dotty.tools.repl.AbstractFileClassLoader.findClass(AbstractFileClassLoader.scala:46)
2025-10-01T17:11:29.0493100Z [error]   at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:592)
2025-10-01T17:11:29.0493908Z [error]   at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:525)
2025-10-01T17:11:29.0495180Z [error]   at dotty.tools.repl.AbstractFileClassLoader.loadClass(AbstractFileClassLoader.scala:60)
2025-10-01T17:11:29.0496144Z [error]   at java.base/java.lang.Class.forName0(Native Method)
2025-10-01T17:11:29.0496825Z [error]   at java.base/java.lang.Class.forName(Class.java:467)
2025-10-01T17:11:29.0497564Z [error]   at dotty.tools.repl.Rendering.pprintRender(Rendering.scala:36)
2025-10-01T17:11:29.0498344Z [error]   at dotty.tools.repl.Rendering.replStringOf(Rendering.scala:86)
2025-10-01T17:11:29.0499132Z [error]   at dotty.tools.repl.Rendering.valueOf$$anonfun$2(Rendering.scala:102)

and again class not found exception in dotty.tools.languageserver.WorksheetTest in test_windows_fast

lihaoyi · 2025-10-02T14:30:50Z

I guess we can check if the class exists in the classloader, and if not then fall back to the previous implementation?

bishabosha · 2025-10-02T16:46:59Z

I pushed a change with the fallback option if the class loader doesn't have the class

lihaoyi · 2025-10-03T16:00:25Z

compiler/src/dotty/tools/repl/Rendering.scala


  var myClassLoader: AbstractFileClassLoader = uninitialized

+  private var pprintState: ((Any, Int, Int, Int) => String) | Null = null


Do we need to cache this? It seems like it would be simpler to reflectively invoke it every time, since it's going to be run once every few seconds/minutes at most performance wouldn't be a problem

i guess i was mostly concerned about catching exceptions every single time in the bad case, maybe it isn't too concerning

Even the overhead of exceptions should be negligible. We're talking a few microseconds for an code path that runs every few seconds, or a 0.0001% slowdown. Definitely not worth the increase in code complexity

I have removed the caching. I was able to inspect the tests e.g. dotty.tools.repl.ReplCompilerTests and those explicitly only have the standard library on the repl's own classpath, so it makes sense for reflection not to find the class. Perhaps a clean solution would be a settings flag to disable pprint functionality (as the fallback is not perfect), or move the pprint stuff to scala.runtime

I guess the question is how is the REPL expected to be used: with the compiler on the classpath, or without? If it's meant to be without, then the only way we can make it work reliably would be to put it in scala-library.jar instead of scala-compiler.jar

its definitely by standard with compiler classpath included, both for 2.13.x and 3.x - so actually its really just these repl specific tests that are restricting the classpath

compiler/src/dotty/tools/repl/Rendering.scala

fixes scala#24111 reflectively load the function each time, (i.e. don't cache). As this is a rare event (i.e user interaction), any cost is less of a concern. if cant load reflectively, then fallback to calling directly.

bishabosha · 2025-10-06T15:36:43Z

now with no caching - meaning there is no complex state management anymore

bishabosha force-pushed the repl-use-pprint-via-reflection branch from ad52fbe to 549dcbd Compare October 1, 2025 16:08

He-Pin reviewed Oct 1, 2025

View reviewed changes

lihaoyi reviewed Oct 1, 2025

View reviewed changes

bishabosha force-pushed the repl-use-pprint-via-reflection branch from 549dcbd to 37a56a6 Compare October 2, 2025 16:45

bishabosha force-pushed the repl-use-pprint-via-reflection branch from 37a56a6 to 350733e Compare October 3, 2025 09:58

SethTisue added this to the 3.8.0 milestone Oct 3, 2025

bishabosha mentioned this pull request Oct 3, 2025

Support //> using dep "..." directives in Scala REPL #24131

Open

lihaoyi reviewed Oct 3, 2025

View reviewed changes

natsukagami reviewed Oct 3, 2025

View reviewed changes

compiler/src/dotty/tools/repl/Rendering.scala Outdated Show resolved Hide resolved

compiler/src/dotty/tools/repl/Rendering.scala Outdated Show resolved Hide resolved

REPL - invoke pprint reflectively

bbbe580

fixes scala#24111 reflectively load the function each time, (i.e. don't cache). As this is a rare event (i.e user interaction), any cost is less of a concern. if cant load reflectively, then fallback to calling directly.

bishabosha force-pushed the repl-use-pprint-via-reflection branch from 350733e to bbbe580 Compare October 6, 2025 15:31


		var myClassLoader: AbstractFileClassLoader = uninitialized

		private var pprintState: ((Any, Int, Int, Int) => String) \| Null = null

REPL - invoke pprint reflectively #24119

Are you sure you want to change the base?

REPL - invoke pprint reflectively #24119

Conversation

bishabosha commented Oct 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bishabosha commented Oct 1, 2025

Uh oh!

bishabosha commented Oct 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

lihaoyi Oct 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

bishabosha commented Oct 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

lihaoyi commented Oct 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bishabosha commented Oct 2, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

bishabosha Oct 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

bishabosha Oct 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

bishabosha Oct 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

bishabosha commented Oct 6, 2025

Uh oh!

Uh oh!

bishabosha commented Oct 1, 2025 •

edited

Loading

bishabosha commented Oct 1, 2025 •

edited

Loading

lihaoyi Oct 1, 2025 •

edited

Loading

bishabosha commented Oct 2, 2025 •

edited

Loading

lihaoyi commented Oct 2, 2025 •

edited

Loading

bishabosha Oct 6, 2025 •

edited

Loading

bishabosha Oct 6, 2025 •

edited

Loading

bishabosha Oct 6, 2025 •

edited

Loading