Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Abstract method implemented by concrete implementation fails in Scala-3.3.1/Scala-Native #9

Open
lihaoyi opened this issue Feb 11, 2024 · 6 comments

Comments

@lihaoyi
Copy link
Member

lihaoyi commented Feb 11, 2024

Running everything serially on c0c771e:

./mill -i -k "unroll[_].tests[_].__.run"                                                                                            

These targets fail:

unroll[3.3.1].tests[abstractClassMethod].v1v2.native.nativeLink 
java.util.NoSuchElementException: key not found: Member(Top(unroll.Unrolled$),D3fooL16java.lang.StringiL16java.lang.StringEO)

unroll[3.3.1].tests[abstractClassMethod].v1v3.native.nativeLink 
java.util.NoSuchElementException: key not found: Member(Top(unroll.Unrolled$),D3fooL16java.lang.StringiL16java.lang.StringEO)

unroll[3.3.1].tests[abstractClassMethod].v2v3.native.nativeLink 
java.util.NoSuchElementException: key not found: Member(Top(unroll.Unrolled$),D3fooL16java.lang.StringizL16java.lang.StringEO)

unroll[3.3.1].tests[abstractTraitMethod].v1v2.native.nativeLink 
java.util.NoSuchElementException: key not found: Member(Top(unroll.Unrolled$),D3fooL16java.lang.StringiL16java.lang.StringEO)

unroll[3.3.1].tests[abstractTraitMethod].v1v3.native.nativeLink 
java.util.NoSuchElementException: key not found: Member(Top(unroll.Unrolled$),D3fooL16java.lang.StringiL16java.lang.StringEO)

unroll[3.3.1].tests[abstractTraitMethod].v2v3.native.nativeLink
java.util.NoSuchElementException: key not found: Member(Top(unroll.Unrolled$),D3fooL16java.lang.StringizL16java.lang.StringEO)

No stack trace is given. Running the targets alone reliably repros the failure.

It appears that somehow the linker is unable to resolve the generated concrete forwarder def foo(s: String, n: Int): String on the object that should be inherited from the abstract class or trait.

Notable it only seems to happen for the "upgrade" test cases, where we're linking an old version of the downstream code against a new version of the upstream code.

The full -Xprint:all is as follows

```scala lihaoyi unroll$ ./mill -i "unroll[3.3.1].tests[abstractClassMethod].v1v2.native.nativeLink" [155/305] unroll[3.3.1].tests[abstractClassMethod].v1.native.compile [info] compiling 1 Scala source to /Users/lihaoyi/Github/unroll/out/unroll/3.3.1/tests/abstractClassMethod/v1/native/compile.dest/classes ... [info] [[syntax trees at end of parser]] // /Users/lihaoyi/Github/unroll/unroll/tests/abstractClassMethod/v1/src/Unrolled.scala [info] package unroll { [info] abstract class Unrolled { [info] def foo(s: String, n: Int = 1): String [info] } [info] module object Unrolled extends Unrolled { [info] def foo(s: String, n: Int = 1) = s + n [info] } [info] class UnrolledCls extends Unrolled { [info] def foo(s: String, n: Int = 1) = s + n [info] } [info] } [info] [[syntax trees at end of typer]] // /Users/lihaoyi/Github/unroll/unroll/tests/abstractClassMethod/v1/src/Unrolled.scala [info] package unroll { [info] abstract class Unrolled() extends Object() { [info] def foo(s: String, n: Int): String [info] def foo$default$2: Int @uncheckedVariance = 1 [info] } [info] final lazy module val Unrolled: unroll.Unrolled = new unroll.Unrolled() [info] final module class Unrolled() extends unroll.Unrolled() { [info] this: unroll.Unrolled.type => [info] def foo(s: String, n: Int): String = s.+(n) [info] def foo$default$2: Int @uncheckedVariance = 1 [info] } [info] class UnrolledCls() extends unroll.Unrolled() { [info] def foo(s: String, n: Int): String = s.+(n) [info] def foo$default$2: Int @uncheckedVariance = 1 [info] } [info] } [info] [[syntax trees at end of inlinedPositions]] // /Users/lihaoyi/Github/unroll/unroll/tests/abstractClassMethod/v1/src/Unrolled.scala: unchanged since typer [info] [[syntax trees at end of sbt-deps]] // /Users/lihaoyi/Github/unroll/unroll/tests/abstractClassMethod/v1/src/Unrolled.scala: unchanged since typer [info] [[syntax trees at end of posttyper]] // /Users/lihaoyi/Github/unroll/unroll/tests/abstractClassMethod/v1/src/Unrolled.scala [info] package unroll { [info] @sourcefile("unroll/tests/abstractClassMethod/v1/src/Unrolled.scala") abstract [info] class Unrolled() extends Object() { [info] def foo(s: String, n: Int): String [info] def foo$default$2: Int @uncheckedVariance = 1 [info] } [info] final lazy module val Unrolled: unroll.Unrolled = new unroll.Unrolled() [info] @sourcefile("unroll/tests/abstractClassMethod/v1/src/Unrolled.scala") final [info] module class Unrolled() extends unroll.Unrolled() { [info] this: unroll.Unrolled.type => [info] private def writeReplace(): AnyRef = [info] new scala.runtime.ModuleSerializationProxy(classOf[unroll.Unrolled.type]) [info] def foo(s: String, n: Int): String = s.+(n) [info] def foo$default$2: Int @uncheckedVariance = 1 [info] } [info] @sourcefile("unroll/tests/abstractClassMethod/v1/src/Unrolled.scala") class [info] UnrolledCls() extends unroll.Unrolled() { [info] def foo(s: String, n: Int): String = s.+(n) [info] def foo$default$2: Int @uncheckedVariance = 1 [info] } [info] } [info] [[syntax trees at end of sbt-api]] // /Users/lihaoyi/Github/unroll/unroll/tests/abstractClassMethod/v1/src/Unrolled.scala: unchanged since posttyper [info] [[syntax trees at end of scalanative-prepareInterop]] // /Users/lihaoyi/Github/unroll/unroll/tests/abstractClassMethod/v1/src/Unrolled.scala: unchanged since posttyper [info] [[syntax trees at end of pickler]] // /Users/lihaoyi/Github/unroll/unroll/tests/abstractClassMethod/v1/src/Unrolled.scala: unchanged since posttyper [info] [[syntax trees at end of unroll]] // /Users/lihaoyi/Github/unroll/unroll/tests/abstractClassMethod/v1/src/Unrolled.scala: unchanged since posttyper [info] [[syntax trees at end of inlining]] // /Users/lihaoyi/Github/unroll/unroll/tests/abstractClassMethod/v1/src/Unrolled.scala: unchanged since posttyper [info] [[syntax trees at end of postInlining]] // /Users/lihaoyi/Github/unroll/unroll/tests/abstractClassMethod/v1/src/Unrolled.scala: unchanged since posttyper [info] [[syntax trees at end of staging]] // /Users/lihaoyi/Github/unroll/unroll/tests/abstractClassMethod/v1/src/Unrolled.scala: unchanged since posttyper [info] [[syntax trees at end of splicing]] // /Users/lihaoyi/Github/unroll/unroll/tests/abstractClassMethod/v1/src/Unrolled.scala: unchanged since posttyper [info] [[syntax trees at end of pickleQuotes]] // /Users/lihaoyi/Github/unroll/unroll/tests/abstractClassMethod/v1/src/Unrolled.scala: unchanged since posttyper [info] [[syntax trees at end of scalanative-prepareInterop-postinline]] // /Users/lihaoyi/Github/unroll/unroll/tests/abstractClassMethod/v1/src/Unrolled.scala: unchanged since posttyper [info] [[syntax trees at end of MegaPhase{crossVersionChecks, firstTransform, checkReentrant, elimPackagePrefixes, cookComments, checkStatic, checkLoopingImplicits, betaReduce, inlineVals, expandSAMs, elimRepeated, refchecks}]] // /Users/lihaoyi/Github/unroll/unroll/tests/abstractClassMethod/v1/src/Unrolled.scala [info] package unroll { [info] @sourcefile("unroll/tests/abstractClassMethod/v1/src/Unrolled.scala") abstract [info] class Unrolled() extends Object() { [info] def foo(s: String, n: Int): String [info] def foo$default$2: Int @uncheckedVariance = 1 [info] } [info] final lazy module val Unrolled: unroll.Unrolled = new unroll.Unrolled() [info] @sourcefile("unroll/tests/abstractClassMethod/v1/src/Unrolled.scala") final [info] module class Unrolled() extends unroll.Unrolled() { [info] private def writeReplace(): AnyRef = [info] new scala.runtime.ModuleSerializationProxy(classOf[unroll.Unrolled.type]) [info] def foo(s: String, n: Int): String = s.+(n) [info] def foo$default$2: Int @uncheckedVariance = 1 [info] } [info] @sourcefile("unroll/tests/abstractClassMethod/v1/src/Unrolled.scala") class [info] UnrolledCls() extends unroll.Unrolled() { [info] def foo(s: String, n: Int): String = s.+(n) [info] def foo$default$2: Int @uncheckedVariance = 1 [info] } [info] } [info] [[syntax trees at end of MegaPhase{protectedAccessors, extmethods, uncacheGivenAliases, elimByName, hoistSuperArgs, forwardDepChecks, specializeApplyMethods, tryCatchPatterns, patternMatcher}]] // /Users/lihaoyi/Github/unroll/unroll/tests/abstractClassMethod/v1/src/Unrolled.scala: unchanged since MegaPhase{crossVersionChecks, firstTransform, checkReentrant, elimPackagePrefixes, cookComments, checkStatic, checkLoopingImplicits, betaReduce, inlineVals, expandSAMs, elimRepeated, refchecks} [info] [[syntax trees at end of preRecheck]] // /Users/lihaoyi/Github/unroll/unroll/tests/abstractClassMethod/v1/src/Unrolled.scala: unchanged since MegaPhase{crossVersionChecks, firstTransform, checkReentrant, elimPackagePrefixes, cookComments, checkStatic, checkLoopingImplicits, betaReduce, inlineVals, expandSAMs, elimRepeated, refchecks} [info] [[syntax trees at end of cc]] // /Users/lihaoyi/Github/unroll/unroll/tests/abstractClassMethod/v1/src/Unrolled.scala: unchanged since MegaPhase{crossVersionChecks, firstTransform, checkReentrant, elimPackagePrefixes, cookComments, checkStatic, checkLoopingImplicits, betaReduce, inlineVals, expandSAMs, elimRepeated, refchecks} [info] [[syntax trees at end of MegaPhase{elimOpaque, explicitOuter, explicitSelf, interpolators, dropBreaks}]] // /Users/lihaoyi/Github/unroll/unroll/tests/abstractClassMethod/v1/src/Unrolled.scala: unchanged since MegaPhase{crossVersionChecks, firstTransform, checkReentrant, elimPackagePrefixes, cookComments, checkStatic, checkLoopingImplicits, betaReduce, inlineVals, expandSAMs, elimRepeated, refchecks} [info] [[syntax trees at end of MegaPhase{pruneErasedDefs, uninitialized, inlinePatterns, vcInlineMethods, seqLiterals, intercepted, getters, specializeFunctions, specializeTuples, liftTry, collectNullableFields, elimOuterSelect, resolveSuper, functionXXLForwarders, paramForwarding, genericTuples, letOverApply, arrayConstructors}]] // /Users/lihaoyi/Github/unroll/unroll/tests/abstractClassMethod/v1/src/Unrolled.scala: unchanged since MegaPhase{crossVersionChecks, firstTransform, checkReentrant, elimPackagePrefixes, cookComments, checkStatic, checkLoopingImplicits, betaReduce, inlineVals, expandSAMs, elimRepeated, refchecks} [info] [[syntax trees at end of erasure]] // /Users/lihaoyi/Github/unroll/unroll/tests/abstractClassMethod/v1/src/Unrolled.scala [info] package unroll { [info] @sourcefile("unroll/tests/abstractClassMethod/v1/src/Unrolled.scala") abstract [info] class Unrolled() extends Object() { [info] def foo(s: String, n: Int): String [info] def foo$default$2(): Int = 1 [info] } [info] final lazy module val Unrolled: unroll.Unrolled = new unroll.Unrolled() [info] @sourcefile("unroll/tests/abstractClassMethod/v1/src/Unrolled.scala") final [info] module class Unrolled() extends unroll.Unrolled() { [info] private def writeReplace(): Object = [info] new scala.runtime.ModuleSerializationProxy(classOf[unroll.Unrolled]) [info] def foo(s: String, n: Int): String = s.+(scala.Int.box(n)) [info] def foo$default$2(): Int = 1 [info] } [info] @sourcefile("unroll/tests/abstractClassMethod/v1/src/Unrolled.scala") class [info] UnrolledCls() extends unroll.Unrolled() { [info] def foo(s: String, n: Int): String = s.+(scala.Int.box(n)) [info] def foo$default$2(): Int = 1 [info] } [info] } [info] [[syntax trees at end of MegaPhase{elimErasedValueType, pureStats, vcElideAllocations, etaReduce, arrayApply, elimPolyFunction, tailrec, completeJavaEnums, mixin, lazyVals, memoize, nonLocalReturns, capturedVars}]] // /Users/lihaoyi/Github/unroll/unroll/tests/abstractClassMethod/v1/src/Unrolled.scala [info] package unroll { [info] @sourcefile("unroll/tests/abstractClassMethod/v1/src/Unrolled.scala") abstract [info] class Unrolled() extends Object { [info] super() [info] def foo(s: String, n: Int): String [info] def foo$default$2(): Int = 1 [info] } [info] final lazy module val Unrolled: unroll.Unrolled = new unroll.Unrolled() [info] @sourcefile("unroll/tests/abstractClassMethod/v1/src/Unrolled.scala") final [info] module class Unrolled() extends unroll.Unrolled { [info] super() [info] private def writeReplace(): Object = [info] new scala.runtime.ModuleSerializationProxy(classOf[unroll.Unrolled]) [info] def foo(s: String, n: Int): String = s.+(scala.Int.box(n)) [info] def foo$default$2(): Int = 1 [info] } [info] @sourcefile("unroll/tests/abstractClassMethod/v1/src/Unrolled.scala") class [info] UnrolledCls() extends unroll.Unrolled { [info] super() [info] def foo(s: String, n: Int): String = s.+(scala.Int.box(n)) [info] def foo$default$2(): Int = 1 [info] } [info] } [info] [[syntax trees at end of constructors]] // /Users/lihaoyi/Github/unroll/unroll/tests/abstractClassMethod/v1/src/Unrolled.scala [info] package unroll { [info] @sourcefile("unroll/tests/abstractClassMethod/v1/src/Unrolled.scala") abstract [info] class Unrolled extends Object { [info] def (): Unit = [info] { [info] super() [info] () [info] } [info] def foo(s: String, n: Int): String [info] def foo$default$2(): Int = 1 [info] } [info] final lazy module val Unrolled: unroll.Unrolled = new unroll.Unrolled() [info] @sourcefile("unroll/tests/abstractClassMethod/v1/src/Unrolled.scala") final [info] module class Unrolled extends unroll.Unrolled { [info] def (): Unit = [info] { [info] super() [info] () [info] } [info] private def writeReplace(): Object = [info] new scala.runtime.ModuleSerializationProxy(classOf[unroll.Unrolled]) [info] def foo(s: String, n: Int): String = s.+(scala.Int.box(n)) [info] def foo$default$2(): Int = 1 [info] } [info] @sourcefile("unroll/tests/abstractClassMethod/v1/src/Unrolled.scala") class [info] UnrolledCls extends unroll.Unrolled { [info] def (): Unit = [info] { [info] super() [info] () [info] } [info] def foo(s: String, n: Int): String = s.+(scala.Int.box(n)) [info] def foo$default$2(): Int = 1 [info] } [info] } [info] [[syntax trees at end of MegaPhase{lambdaLift, elimStaticThis, countOuterAccesses}]] // /Users/lihaoyi/Github/unroll/unroll/tests/abstractClassMethod/v1/src/Unrolled.scala: unchanged since constructors [info] [[syntax trees at end of MegaPhase{dropOuterAccessors, checkNoSuperThis, flatten, transformWildcards, moveStatic, expandPrivate, restoreScopes, selectStatic, Collect entry points, collectSuperCalls, repeatableAnnotations}]] // /Users/lihaoyi/Github/unroll/unroll/tests/abstractClassMethod/v1/src/Unrolled.scala [info] package unroll { [info] @sourcefile("unroll/tests/abstractClassMethod/v1/src/Unrolled.scala") class [info] UnrolledCls extends unroll.Unrolled { [info] def (): Unit = [info] { [info] super() [info] () [info] } [info] def foo(s: String, n: Int): String = s.+(scala.Int.box(n)) [info] def foo$default$2(): Int = 1 [info] } [info] @sourcefile("unroll/tests/abstractClassMethod/v1/src/Unrolled.scala") abstract [info] class Unrolled extends Object { [info] def (): Unit = [info] { [info] super() [info] () [info] } [info] def foo(s: String, n: Int): String [info] def foo$default$2(): Int = 1 [info] } [info] @sourcefile("unroll/tests/abstractClassMethod/v1/src/Unrolled.scala") final [info] module class Unrolled extends unroll.Unrolled { [info] def (): Unit = [info] { [info] super() [info] () [info] } [info] private def writeReplace(): Object = [info] new scala.runtime.ModuleSerializationProxy(classOf[unroll.Unrolled]) [info] def foo(s: String, n: Int): String = s.+(scala.Int.box(n)) [info] def foo$default$2(): Int = 1 [info] } [info] final lazy module val Unrolled: unroll.Unrolled = new unroll.Unrolled() [info] } [info] [[syntax trees at end of scalanative-genNIR]] // /Users/lihaoyi/Github/unroll/unroll/tests/abstractClassMethod/v1/src/Unrolled.scala: unchanged since MegaPhase{dropOuterAccessors, checkNoSuperThis, flatten, transformWildcards, moveStatic, expandPrivate, restoreScopes, selectStatic, Collect entry points, collectSuperCalls, repeatableAnnotations} [info] [[syntax trees at end of genBCode]] // /Users/lihaoyi/Github/unroll/unroll/tests/abstractClassMethod/v1/src/Unrolled.scala: unchanged since MegaPhase{dropOuterAccessors, checkNoSuperThis, flatten, transformWildcards, moveStatic, expandPrivate, restoreScopes, selectStatic, Collect entry points, collectSuperCalls, repeatableAnnotations} [info] done compiling [305/305] unroll[3.3.1].tests[abstractClassMethod].v1v2.native.nativeLink [info] Linking (627 ms) [info] Checking intermediate code (quick) (44 ms) [info] Discovered 688 classes and 3801 methods [info] Optimizing (debug mode) (554 ms) 1 targets failed unroll[3.3.1].tests[abstractClassMethod].v1v2.native.nativeLink java.util.NoSuchElementException: key not found: Member(Top(unroll.Unrolled$),D3fooL16java.lang.StringiL16java.lang.StringEO) ```
@lihaoyi
Copy link
Member Author

lihaoyi commented Feb 11, 2024

@WojciechMazur @densh as the folks most familiar with Scala-Native, do you guys have any tips on how I could investigate this issue? Is there any equivalent to javap -c to visualize the Scala-Native IR?

@WojciechMazur
Copy link

Hey, it looks like a bug in linker of Scala Native, maybe some methods are not reached correctly or we don't emit something in the compiler. I'll try to investigate it based on this branch next week

For javap/scalap alternative you we use either:

  • ScalaNativeP shipped in scala-native-cli:
cs launch org.scala-native:scala-native-cli:<version>  -M scala.scalanative.cli.ScalaNativeP  -- --from-path <path>/x.nir      

It can be used to check a single nir file or NIR defs based on fully qualified name

  • You can also dump NIR after every major phase (classloading, optimizer, lowering), for that modify (config:NativeConfig).withDump(true). In the <scala target directory>/native/ you'll find up to 3 *.hnir files {linked,optimizer,lowered}.hnir containing textual representation of NIR

@lihaoyi
Copy link
Member Author

lihaoyi commented Feb 11, 2024

Ok it seems like the upstream NIR in Scala 3.3.1 which is causing problems has an extra method that the NIR in Scala 2.13.12 does not. Not sure if this is the cause of the misbehavior

 decl @"M15unroll.UnrolledD13foo$default$2iEO" : (@"T15unroll.Unrolled") => int
 
 inlinehint decl @"M15unroll.UnrolledD13foo$default$2iEo" : () => int
 
 decl @"M15unroll.UnrolledD13foo$default$3zEO" : (@"T15unroll.Unrolled") => bool
 
 inlinehint decl @"M15unroll.UnrolledD13foo$default$3zEo" : () => bool
 
 decl @"M15unroll.UnrolledD3fooL16java.lang.StringiL16java.lang.StringEO" : (@"T15unroll.Unrolled", @"T16java.lang.String", int) => @"T16java.lang.String"
 
+inlinehint decl @"M15unroll.UnrolledD3fooL16java.lang.StringiL16java.lang.StringEo" : (@"T16java.lang.String", int) => @"T16java.lang.String"
 
 decl @"M15unroll.UnrolledD3fooL16java.lang.StringizL16java.lang.StringEO" : (@"T15unroll.Unrolled", @"T16java.lang.String", int, bool) => @"T16java.lang.String"
 
 inlinehint decl @"M15unroll.UnrolledD3fooL16java.lang.StringizL16java.lang.StringEo" : (@"T16java.lang.String", int, bool) => @"T16java.lang.String"
 
 decl @"M15unroll.UnrolledRE" : (@"T15unroll.Unrolled") => unit
 
 abstract class @"T15unroll.Unrolled" : @"T16java.lang.Object"

@lihaoyi
Copy link
Member Author

lihaoyi commented Feb 11, 2024

Ok, so I think I've narrowed the crash to the difference between the downstream NIR in 2.13.12 and 3.3.1 below.

2.13.12

lihaoyi unroll$ ~/Downloads/scala-native-cli_3-0.4.17/bin/scala-native-p -v --from-path out/unroll/2.13.12/tests/abstractClassMethod/v1/native/test/compile.dest/classes/unroll/UnrollTestMain\$.nir                                      
Warning: Unknown option -erbose
def @"M22unroll.UnrollTestMain$D4mainLAL16java.lang.String_uEO" : (@"T22unroll.UnrollTestMain$", array[@"T16java.lang.String"]) => unit {
%3(%1 : @"T22unroll.UnrollTestMain$", %2 : array[@"T16java.lang.String"]):
  %4 = classalloc @"T18unroll.UnrolledCls"
  %5 = call[(@"T18unroll.UnrolledCls") => unit] @"M18unroll.UnrolledClsRE" : ptr(%4 : !?@"T18unroll.UnrolledCls")
  %6 = module @"T17unroll.TestUtils$"
  %7 = method %4 : !?@"T18unroll.UnrolledCls", "D13foo$default$2iEO"
  %8 = call[(@"T18unroll.UnrolledCls") => int] %7 : ptr(%4 : !?@"T18unroll.UnrolledCls")
  %9 = method %4 : !?@"T18unroll.UnrolledCls", "D3fooL16java.lang.StringiL16java.lang.StringEO"
  %10 = call[(@"T18unroll.UnrolledCls", @"T16java.lang.String", int) => @"T16java.lang.String"] %9 : ptr(%4 : !?@"T18unroll.UnrolledCls", "cow", %8 : int)
  %11 = method %6 : !?@"T17unroll.TestUtils$", "D19logAssertStartsWithL16java.lang.ObjectL16java.lang.StringuEO"
  %12 = call[(@"T17unroll.TestUtils$", @"T16java.lang.Object", @"T16java.lang.String") => unit] %11 : ptr(%6 : !?@"T17unroll.TestUtils$", %10 : @"T16java.lang.String", "cow1")
  %13 = module @"T17unroll.TestUtils$"
  %14 = method %4 : !?@"T18unroll.UnrolledCls", "D3fooL16java.lang.StringiL16java.lang.StringEO"
  %15 = call[(@"T18unroll.UnrolledCls", @"T16java.lang.String", int) => @"T16java.lang.String"] %14 : ptr(%4 : !?@"T18unroll.UnrolledCls", "cow", int 2)
  %16 = method %13 : !?@"T17unroll.TestUtils$", "D19logAssertStartsWithL16java.lang.ObjectL16java.lang.StringuEO"
  %17 = call[(@"T17unroll.TestUtils$", @"T16java.lang.Object", @"T16java.lang.String") => unit] %16 : ptr(%13 : !?@"T17unroll.TestUtils$", %15 : @"T16java.lang.String", "cow2")
  %18 = module @"T17unroll.TestUtils$"
  %19 = module @"T16unroll.Unrolled$"
  %20 = module @"T16unroll.Unrolled$"
  %21 = method %20 : !?@"T16unroll.Unrolled$", "D13foo$default$2iEO"
  %22 = call[(@"T16unroll.Unrolled$") => int] %21 : ptr(%20 : !?@"T16unroll.Unrolled$")
  %23 = method %19 : !?@"T16unroll.Unrolled$", "D3fooL16java.lang.StringiL16java.lang.StringEO"
  %24 = call[(@"T16unroll.Unrolled$", @"T16java.lang.String", int) => @"T16java.lang.String"] %23 : ptr(%19 : !?@"T16unroll.Unrolled$", "cow", %22 : int)
  %25 = method %18 : !?@"T17unroll.TestUtils$", "D19logAssertStartsWithL16java.lang.ObjectL16java.lang.StringuEO"
  %26 = call[(@"T17unroll.TestUtils$", @"T16java.lang.Object", @"T16java.lang.String") => unit] %25 : ptr(%18 : !?@"T17unroll.TestUtils$", %24 : @"T16java.lang.String", "cow1")
  %27 = module @"T17unroll.TestUtils$"
  %28 = module @"T16unroll.Unrolled$"
  %29 = method %28 : !?@"T16unroll.Unrolled$", "D3fooL16java.lang.StringiL16java.lang.StringEO"
  %30 = call[(@"T16unroll.Unrolled$", @"T16java.lang.String", int) => @"T16java.lang.String"] %29 : ptr(%28 : !?@"T16unroll.Unrolled$", "cow", int 2)
  %31 = method %27 : !?@"T17unroll.TestUtils$", "D19logAssertStartsWithL16java.lang.ObjectL16java.lang.StringuEO"
  %32 = call[(@"T17unroll.TestUtils$", @"T16java.lang.Object", @"T16java.lang.String") => unit] %31 : ptr(%27 : !?@"T17unroll.TestUtils$", %30 : @"T16java.lang.String", "cow2")
  ret %32 : unit
}

def @"M22unroll.UnrollTestMain$RE" : (@"T22unroll.UnrollTestMain$") => unit {
%2(%1 : @"T22unroll.UnrollTestMain$"):
  %3 = call[(@"T16java.lang.Object") => unit] @"M16java.lang.ObjectRE" : ptr(%1 : @"T22unroll.UnrollTestMain$")
  ret unit
}

module @"T22unroll.UnrollTestMain$" : @"T16java.lang.Object"

3.3.1

lihaoyi unroll$ ~/Downloads/scala-native-cli_3-0.4.17/bin/scala-native-p -v --from-path  out/unroll/3.3.1/tests/abstractClassMethod/v1/native/test/compile.dest/classes/unroll/UnrollTestMain\$.nir 
Warning: Unknown option -erbose
def @"M22unroll.UnrollTestMain$D12writeReplaceL16java.lang.ObjectEPT22unroll.UnrollTestMain$" : (@"T22unroll.UnrollTestMain$") => @"T16java.lang.Object" {
%2(%1 : @"T22unroll.UnrollTestMain$"):
  %3 = classalloc @"T38scala.runtime.ModuleSerializationProxy"
  %4 = call[(@"T38scala.runtime.ModuleSerializationProxy", @"T15java.lang.Class") => unit] @"M38scala.runtime.ModuleSerializationProxyRL15java.lang.ClassE" : ptr(%3 : !?@"T38scala.runtime.ModuleSerializationProxy", classOf[@"T22unroll.UnrollTestMain$"])
  ret %3 : !?@"T38scala.runtime.ModuleSerializationProxy"
}

def @"M22unroll.UnrollTestMain$D4mainLAL16java.lang.String_uEO" : (@"T22unroll.UnrollTestMain$", array[@"T16java.lang.String"]) => unit {
%3(%2 : @"T22unroll.UnrollTestMain$", %1 : array[@"T16java.lang.String"]):
  %4 = classalloc @"T18unroll.UnrolledCls"
  %5 = call[(@"T18unroll.UnrolledCls") => unit] @"M18unroll.UnrolledClsRE" : ptr(%4 : !?@"T18unroll.UnrolledCls")
  %6 = module @"T17unroll.TestUtils$"
  %7 = method %4 : !?@"T18unroll.UnrolledCls", "D13foo$default$2iEO"
  %8 = call[(@"T18unroll.UnrolledCls") => int] %7 : ptr(%4 : !?@"T18unroll.UnrolledCls")
  %9 = method %4 : !?@"T18unroll.UnrolledCls", "D3fooL16java.lang.StringiL16java.lang.StringEO"
  %10 = call[(@"T18unroll.UnrolledCls", @"T16java.lang.String", int) => @"T16java.lang.String"] %9 : ptr(%4 : !?@"T18unroll.UnrolledCls", "cow", %8 : int)
  %11 = call[(@"T17unroll.TestUtils$", @"T16java.lang.Object", @"T16java.lang.String") => unit] @"M17unroll.TestUtils$D19logAssertStartsWithL16java.lang.ObjectL16java.lang.StringuEO" : ptr(%6 : !?@"T17unroll.TestUtils$", %10 : @"T16java.lang.String", "cow1")
  %12 = module @"T17unroll.TestUtils$"
  %13 = method %4 : !?@"T18unroll.UnrolledCls", "D3fooL16java.lang.StringiL16java.lang.StringEO"
  %14 = call[(@"T18unroll.UnrolledCls", @"T16java.lang.String", int) => @"T16java.lang.String"] %13 : ptr(%4 : !?@"T18unroll.UnrolledCls", "cow", int 2)
  %15 = call[(@"T17unroll.TestUtils$", @"T16java.lang.Object", @"T16java.lang.String") => unit] @"M17unroll.TestUtils$D19logAssertStartsWithL16java.lang.ObjectL16java.lang.StringuEO" : ptr(%12 : !?@"T17unroll.TestUtils$", %14 : @"T16java.lang.String", "cow2")
  %16 = module @"T17unroll.TestUtils$"
  %17 = module @"T16unroll.Unrolled$"
  %18 = module @"T16unroll.Unrolled$"
  %19 = call[(@"T16unroll.Unrolled$") => int] @"M16unroll.Unrolled$D13foo$default$2iEO" : ptr(%18 : !?@"T16unroll.Unrolled$")
  %20 = call[(@"T16unroll.Unrolled$", @"T16java.lang.String", int) => @"T16java.lang.String"] @"M16unroll.Unrolled$D3fooL16java.lang.StringiL16java.lang.StringEO" : ptr(%17 : !?@"T16unroll.Unrolled$", "cow", %19 : int)
  %21 = call[(@"T17unroll.TestUtils$", @"T16java.lang.Object", @"T16java.lang.String") => unit] @"M17unroll.TestUtils$D19logAssertStartsWithL16java.lang.ObjectL16java.lang.StringuEO" : ptr(%16 : !?@"T17unroll.TestUtils$", %20 : @"T16java.lang.String", "cow1")
  %22 = module @"T17unroll.TestUtils$"
  %23 = module @"T16unroll.Unrolled$"
  %24 = call[(@"T16unroll.Unrolled$", @"T16java.lang.String", int) => @"T16java.lang.String"] @"M16unroll.Unrolled$D3fooL16java.lang.StringiL16java.lang.StringEO" : ptr(%23 : !?@"T16unroll.Unrolled$", "cow", int 2)
  %25 = call[(@"T17unroll.TestUtils$", @"T16java.lang.Object", @"T16java.lang.String") => unit] @"M17unroll.TestUtils$D19logAssertStartsWithL16java.lang.ObjectL16java.lang.StringuEO" : ptr(%22 : !?@"T17unroll.TestUtils$", %24 : @"T16java.lang.String", "cow2")
  ret unit
}

def @"M22unroll.UnrollTestMain$RE" : (@"T22unroll.UnrollTestMain$") => unit {
%2(%1 : @"T22unroll.UnrollTestMain$"):
  %3 = call[(@"T16java.lang.Object") => unit] @"M16java.lang.ObjectRE" : ptr(%1 : @"T22unroll.UnrollTestMain$")
  ret unit
}

module @"T22unroll.UnrollTestMain$" : @"T16java.lang.Object"

Swapping in the 2.13.12 downstream NIR to replace the 3.3.1 downstream NIR seems to make linking complete successfully, even if the 3.3.1 upstream NIR is unchanged.

It's unclear to me what the significant difference is. The NIR seems to have changed a lot between 2.13.12 and 3.3.1

The console log below demonstrates the fact that replacing the 3.3.1 version of the downstream NIR with the 2.13.12 version is the only thing that makes linking succeed; replacing the upstream NIR does not.

lihaoyi unroll$ ./mill -i "unroll[{2.13.12,3.3.1}].tests[abstractClassMethod].v1v2.native.nativeLink"                                                                                               
...
[577/577] unroll[3.3.1].tests[abstractClassMethod].v1v2.native.nativeLink 
[info] Linking (338 ms)
[info] Checking intermediate code (quick) (13 ms)
[info] Discovered 688 classes and 3801 methods
[info] Optimizing (debug mode) (283 ms)
1 targets failed
unroll[3.3.1].tests[abstractClassMethod].v1v2.native.nativeLink java.util.NoSuchElementException: key not found: Member(Top(unroll.Unrolled$),D3fooL16java.lang.StringiL16java.lang.StringEO)

lihaoyi unroll$ mv out/unroll/2.13.12/tests/abstractClassMethod/v1/native/test/compile.dest/classes/unroll/UnrollTestMain\$.nir out/unroll/3.3.1/tests/abstractClassMethod/v1/native/test/compile.dest/classes/unroll/UnrollTestMain\$.nir
lihaoyi unroll$ cp /Users/lihaoyi/Github/unroll/out/unroll/2.13.12/tests/abstractClassMethod/v22/native/jar.dest/out.jar /Users/lihaoyi/Github/unroll/out/unroll/3.3.1/tests/abstractClassMethod/v1/native/test/jar.dest/out.jar
lihaoyi unroll$ mv out/unroll/2.13.12/tests/abstractClassMethod/v2/native/compile.dest/classes/unroll/Unrolled.nir out/unroll/3.3.1/tests/abstractClassMethod/v2/native/compile.dest/classes/unroll/Unrolled.nir        
lihaoyi unroll$ ./mill -i "unroll[{2.13.12,3.3.1}].tests[abstractClassMethod].v1v2.native.nativeLink"                                                                                                           
[577/577] unroll[3.3.1].tests[abstractClassMethod].v1v2.native.nativeLink 
[info] Linking (645 ms)
[info] Checking intermediate code (quick) (45 ms)
[info] Discovered 688 classes and 3801 methods
[info] Optimizing (debug mode) (609 ms)
1 targets failed
unroll[3.3.1].tests[abstractClassMethod].v1v2.native.nativeLink java.util.NoSuchElementException: key not found: Member(Top(unroll.Unrolled$),D3fooL16java.lang.StringiL16java.lang.StringEO)

lihaoyi unroll$ mv out/unroll/2.13.12/tests/abstractClassMethod/v2/native/compile.dest/classes/unroll/Unrolled\$.nir out/unroll/3.3.1/tests/abstractClassMethod/v2/native/compile.dest/classes/unroll/Unrolled\$.nir

lihaoyi unroll$ ./mill -i "unroll[{2.13.12,3.3.1}].tests[abstractClassMethod].v1v2.native.nativeLink"                                                                                                               
[577/577] unroll[3.3.1].tests[abstractClassMethod].v1v2.native.nativeLink 
[info] Linking (615 ms)
[info] Checking intermediate code (quick) (46 ms)
[info] Discovered 688 classes and 3801 methods
[info] Optimizing (debug mode) (552 ms)
1 targets failed
unroll[3.3.1].tests[abstractClassMethod].v1v2.native.nativeLink java.util.NoSuchElementException: key not found: Member(Top(unroll.Unrolled$),D3fooL16java.lang.StringiL16java.lang.StringEO)

lihaoyi unroll$ mv out/unroll/2.13.12/tests/abstractClassMethod/v2/native/compile.dest/classes/unroll/UnrolledCls.nir out/unroll/3.3.1/tests/abstractClassMethod/v2/native/compile.dest/classes/unroll/UnrolledCls.nir

lihaoyi unroll$ ./mill -i "unroll[{2.13.12,3.3.1}].tests[abstractClassMethod].v1v2.native.nativeLink"                                                                                                                 
[577/577] unroll[3.3.1].tests[abstractClassMethod].v1v2.native.nativeLink 
[info] Linking (606 ms)
[info] Checking intermediate code (quick) (54 ms)
[info] Discovered 688 classes and 3801 methods
[info] Optimizing (debug mode) (576 ms)
1 targets failed
unroll[3.3.1].tests[abstractClassMethod].v1v2.native.nativeLink java.util.NoSuchElementException: key not found: Member(Top(unroll.Unrolled$),D3fooL16java.lang.StringiL16java.lang.StringEO)

lihaoyi unroll$ mv out/unroll/2.13.12/tests/abstractClassMethod/v1/native/test/compile.dest/classes/unroll/UnrollTestMain\$.nir out/unroll/3.3.1/tests/abstractClassMethod/v1/native/test/compile.dest/classes/unroll/UnrollTestMain\$.nir

lihaoyi unroll$ ./mill -i "unroll[{2.13.12,3.3.1}].tests[abstractClassMethod].v1v2.native.nativeLink"                                                                                                                                     
[577/577] unroll[3.3.1].tests[abstractClassMethod].v1v2.native.nativeLink 
[info] Linking (626 ms)
[info] Checking intermediate code (quick) (45 ms)
[info] Discovered 688 classes and 3801 methods
[info] Optimizing (debug mode) (560 ms)
[info] Generating intermediate code (610 ms)
[info] Produced 10 files
[info] Compiling to native code (1502 ms)
[info] Total (3419 ms)

@WojciechMazur
Copy link

To fix it we'll need to modify how we generate method calls for statically known methods. In Scala 2 we've always did created a method accessor and then called it - optimizer would typically know that there is only 1 available target for such method and would replace it with a static call.
For Scala 3, there probably there was an optimization/bug which have caused to always emit a statically known method call directly. We'll fix it and ship it in the next release (probably this week)

@lihaoyi
Copy link
Member Author

lihaoyi commented Feb 12, 2024

Thanks @WojciechMazur !

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants