Make it a fatal error if erasure cannot resolve a type #16373

odersky · 2022-11-18T11:53:35Z

Fixes #16368 (?)

It would be good if somebody else would follow up here. I have just identified and avoided the immediate source of the crash, but I don't know what else could go wrong. In any case it's probably just a matter of surviving non-existing types.

jchyb · 2022-11-18T15:27:28Z

I have run the scala-cli minimization with a locally published compiler from this PR and it still returns the same assertion error unfortunately:

java.lang.AssertionError: assertion failed
Error compiling project (Scala 3.3.0-RC1-bin-SNAPSHOT, JVM)
        at scala.runtime.Scala3RunTime$.assertFailed(Scala3RunTime.scala:11)
        at dotty.tools.dotc.core.Types$MethodType.<init>(Types.scala:3883)
        at dotty.tools.dotc.core.Types$CachedMethodType.<init>(Types.scala:3903)
        at dotty.tools.dotc.core.Types$MethodTypeCompanion.apply(Types.scala:3974)
        at dotty.tools.dotc.core.Types$MethodTypeCompanion.apply(Types.scala:3973)
        at dotty.tools.dotc.core.Types$LambdaTypeCompanion.apply(Types.scala:3915)
        at dotty.tools.dotc.core.TypeErasure.eraseInfo(TypeErasure.scala:740)
        at dotty.tools.dotc.core.TypeErasure$.transformInfo(TypeErasure.scala:244)
        at dotty.tools.dotc.transform.Erasure.transform(Erasure.scala:97)
        at dotty.tools.dotc.core.Denotations$SingleDenotation.goForward$1(Denotations.scala:833)
        at dotty.tools.dotc.core.Denotations$SingleDenotation.current(Denotations.scala:882)
        at dotty.tools.dotc.core.Denotations$SingleDenotation.goForward$1(Denotations.scala:853)
        at dotty.tools.dotc.core.Denotations$SingleDenotation.current(Denotations.scala:882)
        at dotty.tools.dotc.core.Denotations$SingleDenotation.goForward$1(Denotations.scala:853)
        at dotty.tools.dotc.core.Denotations$SingleDenotation.current(Denotations.scala:882)
        at dotty.tools.dotc.core.Denotations$SingleDenotation.goForward$1(Denotations.scala:853)
        at dotty.tools.dotc.core.Denotations$SingleDenotation.current(Denotations.scala:882)
        at dotty.tools.dotc.core.Denotations$SingleDenotation.goForward$1(Denotations.scala:853)
        at dotty.tools.dotc.core.Denotations$SingleDenotation.current(Denotations.scala:882)
        at dotty.tools.dotc.core.Denotations$SingleDenotation.goForward$1(Denotations.scala:853)
        at dotty.tools.dotc.core.Denotations$SingleDenotation.current(Denotations.scala:882)
        at dotty.tools.dotc.core.Denotations$SingleDenotation.goForward$1(Denotations.scala:853)
        at dotty.tools.dotc.core.Denotations$SingleDenotation.current(Denotations.scala:882)
        at dotty.tools.dotc.core.Denotations$SingleDenotation.goForward$1(Denotations.scala:853)
        at dotty.tools.dotc.core.Denotations$SingleDenotation.current(Denotations.scala:882)
        at dotty.tools.dotc.core.Denotations$SingleDenotation.goForward$1(Denotations.scala:853)
        at dotty.tools.dotc.core.Denotations$SingleDenotation.current(Denotations.scala:882)
        at dotty.tools.dotc.core.Denotations$SingleDenotation.goForward$1(Denotations.scala:853)
        at dotty.tools.dotc.core.Denotations$SingleDenotation.current(Denotations.scala:882)
        at dotty.tools.dotc.core.Denotations$SingleDenotation.goForward$1(Denotations.scala:853)
        at dotty.tools.dotc.core.Denotations$SingleDenotation.current(Denotations.scala:882)
        at dotty.tools.dotc.core.Denotations$SingleDenotation.goForward$1(Denotations.scala:853)
        at dotty.tools.dotc.core.Denotations$SingleDenotation.current(Denotations.scala:882)
        at dotty.tools.dotc.core.Denotations$SingleDenotation.goForward$1(Denotations.scala:853)
        at dotty.tools.dotc.core.Denotations$SingleDenotation.current(Denotations.scala:882)
        at dotty.tools.dotc.core.Denotations$SingleDenotation.goForward$1(Denotations.scala:853)
        at dotty.tools.dotc.core.Denotations$SingleDenotation.current(Denotations.scala:882)
        at dotty.tools.dotc.core.Denotations$SingleDenotation.goForward$1(Denotations.scala:853)
        at dotty.tools.dotc.core.Denotations$SingleDenotation.current(Denotations.scala:882)
        at dotty.tools.dotc.core.Denotations$SingleDenotation.goForward$1(Denotations.scala:853)
        at dotty.tools.dotc.core.Denotations$SingleDenotation.current(Denotations.scala:882)
        at dotty.tools.dotc.core.Denotations$SingleDenotation.goForward$1(Denotations.scala:853)
        at dotty.tools.dotc.core.Denotations$SingleDenotation.current(Denotations.scala:882)
        at dotty.tools.dotc.core.Denotations$SingleDenotation.goForward$1(Denotations.scala:853)
        at dotty.tools.dotc.core.Denotations$SingleDenotation.current(Denotations.scala:882)
        at dotty.tools.dotc.core.Denotations$SingleDenotation.goForward$1(Denotations.scala:853)
        at dotty.tools.dotc.core.Denotations$SingleDenotation.current(Denotations.scala:882)
        at dotty.tools.dotc.core.Denotations$SingleDenotation.goForward$1(Denotations.scala:853)
        at dotty.tools.dotc.core.Denotations$SingleDenotation.current(Denotations.scala:882)
        at dotty.tools.dotc.core.Symbols$Symbol.recomputeDenot(Symbols.scala:120)
        at dotty.tools.dotc.core.Symbols$Symbol.computeDenot(Symbols.scala:114)
        at dotty.tools.dotc.core.Symbols$Symbol.denot(Symbols.scala:107)
        at dotty.tools.dotc.core.Symbols$.toDenot(Symbols.scala:494)
        at dotty.tools.dotc.transform.Mixin.needsTraitSetter(Mixin.scala:174)
        at dotty.tools.dotc.transform.Mixin.transformSym$$anonfun$1(Mixin.scala:153)
        at scala.runtime.function.JProcedure1.apply(JProcedure1.java:15)
        at scala.runtime.function.JProcedure1.apply(JProcedure1.java:10)
        at scala.collection.immutable.List.foreach(List.scala:333)
        at dotty.tools.dotc.core.Scopes$Scope.foreach(Scopes.scala:93)
        at dotty.tools.dotc.transform.Mixin.transformSym(Mixin.scala:157)
        at dotty.tools.dotc.core.DenotTransformers$SymTransformer.transform(DenotTransformers.scala:70)
        at dotty.tools.dotc.core.DenotTransformers$SymTransformer.transform$(DenotTransformers.scala:65)
        at dotty.tools.dotc.transform.Mixin.transform(Mixin.scala:113)
        at dotty.tools.dotc.core.Denotations$SingleDenotation.goForward$1(Denotations.scala:833)
        at dotty.tools.dotc.core.Denotations$SingleDenotation.current(Denotations.scala:882)
        at dotty.tools.dotc.core.Denotations$SingleDenotation.goForward$1(Denotations.scala:853)
        at dotty.tools.dotc.core.Denotations$SingleDenotation.current(Denotations.scala:882)
        at dotty.tools.dotc.core.Denotations$SingleDenotation.goForward$1(Denotations.scala:853)
        at dotty.tools.dotc.core.Denotations$SingleDenotation.current(Denotations.scala:882)
        at dotty.tools.dotc.core.Denotations$SingleDenotation.goForward$1(Denotations.scala:853)
        at dotty.tools.dotc.core.Denotations$SingleDenotation.current(Denotations.scala:882)
        at dotty.tools.dotc.core.Denotations$SingleDenotation.goForward$1(Denotations.scala:853)
        at dotty.tools.dotc.core.Denotations$SingleDenotation.current(Denotations.scala:882)
        at dotty.tools.dotc.core.Symbols$Symbol.recomputeDenot(Symbols.scala:120)
        at dotty.tools.dotc.core.Symbols$Symbol.computeDenot(Symbols.scala:114)
        at dotty.tools.dotc.core.Symbols$Symbol.denot(Symbols.scala:107)
        at dotty.tools.dotc.core.Symbols$.toDenot(Symbols.scala:494)
        at dotty.tools.dotc.core.SymDenotations$ClassDenotation.registeredCompanion(SymDenotations.scala:2328)
        at dotty.tools.dotc.core.SymDenotations$SymDenotation.companionType(SymDenotations.scala:1220)
        at dotty.tools.dotc.core.SymDenotations$SymDenotation.companionClass(SymDenotations.scala:1228)
        at dotty.tools.dotc.core.SymDenotations$SymDenotation.linkedClass(SymDenotations.scala:1270)
        at dotty.tools.backend.jvm.BTypesFromSymbols.setClassInfo(BTypesFromSymbols.scala:175)
        at dotty.tools.backend.jvm.BTypesFromSymbols.classBTypeFromSymbol$$anonfun$1(BTypesFromSymbols.scala:91)
        at scala.collection.mutable.HashMap.getOrElse(HashMap.scala:436)
        at dotty.tools.backend.jvm.BTypesFromSymbols.classBTypeFromSymbol(BTypesFromSymbols.scala:92)
        at dotty.tools.backend.jvm.BCodeHelpers$BCInnerClassGen.getClassBType(BCodeHelpers.scala:260)
        at dotty.tools.backend.jvm.BCodeHelpers$BCInnerClassGen.getClassBType$(BCodeHelpers.scala:212)
        at dotty.tools.backend.jvm.BCodeHelpers$JCommonBuilder.getClassBType(BCodeHelpers.scala:653)
        at dotty.tools.backend.jvm.BCodeHelpers.primitiveOrClassToBType$1$$anonfun$1(BCodeHelpers.scala:797)
        at scala.collection.immutable.HashMap.getOrElse(HashMap.scala:678)
        at dotty.tools.backend.jvm.BCodeHelpers.primitiveOrClassToBType$1(BCodeHelpers.scala:797)
        at dotty.tools.backend.jvm.BCodeHelpers.dotty$tools$backend$jvm$BCodeHelpers$$typeToTypeKind(BCodeHelpers.scala:816)
        at dotty.tools.backend.jvm.BCodeHelpers$BCInnerClassGen.toTypeKind(BCodeHelpers.scala:284)
        at dotty.tools.backend.jvm.BCodeHelpers$BCInnerClassGen.toTypeKind$(BCodeHelpers.scala:212)
        at dotty.tools.backend.jvm.BCodeHelpers$JCommonBuilder.toTypeKind(BCodeHelpers.scala:653)
        at dotty.tools.backend.jvm.BCodeHelpers$BCForwardersGen.addForwarder(BCodeHelpers.scala:517)
        at dotty.tools.backend.jvm.BCodeHelpers$BCForwardersGen.addForwarders$$anonfun$3(BCodeHelpers.scala:593)
        at scala.runtime.function.JProcedure1.apply(JProcedure1.java:15)
        at scala.runtime.function.JProcedure1.apply(JProcedure1.java:10)
        at scala.collection.immutable.List.foreach(List.scala:333)
        at dotty.tools.backend.jvm.BCodeHelpers$BCForwardersGen.addForwarders(BCodeHelpers.scala:595)
        at dotty.tools.backend.jvm.BCodeHelpers$BCForwardersGen.addForwarders$(BCodeHelpers.scala:489)
        at dotty.tools.backend.jvm.BCodeHelpers$JCommonBuilder.addForwarders(BCodeHelpers.scala:653)
        at dotty.tools.backend.jvm.BCodeHelpers$JMirrorBuilder.genMirrorClass(BCodeHelpers.scala:700)
        at dotty.tools.backend.jvm.GenBCodePipeline$Worker1.visit(GenBCode.scala:257)
        at dotty.tools.backend.jvm.GenBCodePipeline$Worker1.run(GenBCode.scala:231)
        at dotty.tools.backend.jvm.GenBCodePipeline.buildAndSendToDisk(GenBCode.scala:598)
        at dotty.tools.backend.jvm.GenBCodePipeline.run(GenBCode.scala:564)
        at dotty.tools.backend.jvm.GenBCode.run(GenBCode.scala:69)
        at dotty.tools.dotc.core.Phases$Phase.runOn$$anonfun$1(Phases.scala:324)
        at scala.collection.immutable.List.map(List.scala:246)
        at dotty.tools.dotc.core.Phases$Phase.runOn(Phases.scala:328)
        at dotty.tools.backend.jvm.GenBCode.runOn(GenBCode.scala:77)
        at dotty.tools.dotc.Run.runPhases$1$$anonfun$1(Run.scala:244)
        at scala.runtime.function.JProcedure1.apply(JProcedure1.java:15)
        at scala.runtime.function.JProcedure1.apply(JProcedure1.java:10)
        at scala.collection.ArrayOps$.foreach$extension(ArrayOps.scala:1321)
        at dotty.tools.dotc.Run.runPhases$1(Run.scala:260)
        at dotty.tools.dotc.Run.compileUnits$$anonfun$1(Run.scala:268)
        at dotty.tools.dotc.Run.compileUnits$$anonfun$adapted$1(Run.scala:277)
        at dotty.tools.dotc.util.Stats$.maybeMonitored(Stats.scala:68)
        at dotty.tools.dotc.Run.compileUnits(Run.scala:277)
        at dotty.tools.dotc.Run.compileSources(Run.scala:195)
        at dotty.tools.dotc.Run.compile(Run.scala:179)
        at dotty.tools.dotc.Driver.doCompile(Driver.scala:35)
        at dotty.tools.xsbt.CompilerBridgeDriver.run(CompilerBridgeDriver.java:88)
        at dotty.tools.xsbt.CompilerBridge.run(CompilerBridge.java:22)
        at sbt.internal.inc.AnalyzingCompiler.compile(AnalyzingCompiler.scala:91)
        at sbt.internal.inc.bloop.internal.BloopHighLevelCompiler.compileSources$1(BloopHighLevelCompiler.scala:133)
        at sbt.internal.inc.bloop.internal.BloopHighLevelCompiler.$anonfun$compile$7(BloopHighLevelCompiler.scala:151)
        at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
        at sbt.internal.inc.bloop.internal.BloopHighLevelCompiler.$anonfun$compile$1(BloopHighLevelCompiler.scala:71)
        at bloop.tracing.BraveTracer.traceInternal(BraveTracer.scala:65)
        at bloop.tracing.BraveTracer.trace(BraveTracer.scala:39)
        at sbt.internal.inc.bloop.internal.BloopHighLevelCompiler.timed$1(BloopHighLevelCompiler.scala:70)
        at sbt.internal.inc.bloop.internal.BloopHighLevelCompiler.$anonfun$compile$6(BloopHighLevelCompiler.scala:151)
        at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
        at monix.eval.internal.TaskRunLoop$.monix$eval$internal$TaskRunLoop$$loop$1(TaskRunLoop.scala:187)
        at monix.eval.internal.TaskRunLoop$RestartCallback$1.onSuccess(TaskRunLoop.scala:119)
        at monix.eval.Task$.$anonfun$forkedUnit$2(Task.scala:1463)
        at java.base/java.util.concurrent.ForkJoinTask$RunnableExecuteAction.exec(ForkJoinTask.java:1395)
        at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373)
        at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182)
        at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655)
        at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622)
        at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165)
Error: Unexpected error when compiling project_2dfd871e55_2dfd871e55-8af90d9c52: 'assertion failed'

odersky · 2022-11-18T16:57:04Z

OK, so this did not work. We need to dig deeper at the point whete it happened:

   else if rt.exists then 
          MethodType(Nil, Nil,
            eraseResult(rt.translateFromRepeated(toArray = sourceLanguage.isJava)))
        else rt

So, rt does exist, and the test I added is not needed. But the result of eraseResult is NoType, or otherwise the assert in MethodType would not trigger. So we need to find out where the NoType arises. Can you give it a try?

jchyb · 2022-11-21T15:28:42Z

Sure, I'll try! (sorry, I did not notice your comment before, I hope it's not too late)

odersky · 2022-11-21T15:40:16Z

No, it's all yours :-)

jchyb · 2022-11-28T10:08:25Z

So it looks like the problematic rt types are akka.stream.scaladsl.Source[akka.util.ByteString, AnyRef], and akka.stream.javadsl.Source[akka.util.ByteString, AnyRef]. Both obviously have missing definitions on the class path here.

TypeErasure through a chain of pattern matches would return rt.translucentSuperType, which led to the problematic NoType. Indirectly, rt.typeSymbol would return NoSymbol (despite the fact that rt exists), further invalidating the pattern match chain.
Similarly to the attempt above I tried to just return the rt directly which resulted in a „successful” compilation, but running the program would give us an even worse runtime error:

Exception in thread "main" java.lang.NoClassDefFoundError: akka/actor/ActorRefFactory
        at akka.http.scaladsl.model.MediaType.<init>(MediaType.scala:41)
        at akka.http.scaladsl.model.MediaType$Binary.<init>(MediaType.scala:209)
        at akka.http.scaladsl.model.MediaType$$anon$11.<init>(MediaType.scala:132)
        at akka.http.scaladsl.model.MediaType$.customBinary(MediaType.scala:140)
        at akka.http.scaladsl.model.MediaTypes$.<clinit>(MediaType.scala:345)
        at akka.http.scaladsl.model.ContentTypes$.<clinit>(ContentType.scala:126)
        at akka.http.scaladsl.model.HttpEntity$.<clinit>(HttpEntity.scala:332)
        at tu$package$.main(tu.scala:6)
        at main.main(tu.scala:5)

My best solution so far is to just throw an error directly from the eraseInfo method, like this:

else {
  val erased = eraseResult(rt.translateFromRepeated(toArray = sourceLanguage.isJava))
        
  if !erased.exists then
    // Related to issue #16368
    // When compiling with an incomplete classpath, due to the erased type not existing, an 
    // assertion inside MethodType constructor would fail, so instead we return 
    // NoType explicitly to continue compilation and report the error.
    report.error(
       i"""cannot resolve reference to type ${rt.show}
          |the classfile defining the type might be missing from the classpath""".toMessage, 
       sym.srcPos
    )
    NoType
  else
    MethodType(Nil, Nil, erased)
}

But the missing typeSymbol hints at the actual error hiding somewhere else.

odersky · 2022-12-07T18:18:21Z

@jchyb I think that approach could work. It's just that sym.srcPos would probably non-sensical since the symbol is not in our own sources.

Maybe yould could try to throw a MissingType exception instead? But I am not sure it would be caught in the backend.

odersky · 2022-12-09T10:14:18Z

See my comment on #16368. I think we need to diagnose better why the error appears in cli and sbt in the first place.

odersky · 2022-12-09T10:20:12Z

I think the problem is that an error should have been reported before (and is reported in my attempt to reproduce it). So why is that error missing?

I think it's a hopeless cause to try to make the backend survive for programs that are incorrect. That will never work. Instead, we must make sure the backend is not invoked when the program has errors. Only in this case it seems that in some situation the error is either not emitted or suppressed afterwards.

Fixes scala#16368

odersky · 2022-12-09T13:50:49Z

I think I have a robust fix now. We need to issue a FatalError in order to exit from the backend without causing further crashes.

mdedetrich · 2023-09-15T14:42:04Z

There is a suspicion that this change has caused a regression when it comes to undefined type parameters, you can look at #18555 for more details.

odersky closed this Nov 18, 2022

odersky reopened this Nov 18, 2022

odersky assigned jchyb Nov 21, 2022

Make it a fatal error if erasure cannot resolve a type

bfb8a9d

Fixes scala#16368

odersky force-pushed the fix-16368 branch from efb38d2 to bfb8a9d Compare December 9, 2022 13:48

odersky changed the title ~~Avoid immediate source of crash in TypeErasure for #16368~~ Make it a fatal error if erasure cannot resolve a type Dec 9, 2022

odersky requested a review from jchyb December 9, 2022 13:51

Kordyjan modified the milestones: Future versions, 3.3.0-RC1 Dec 12, 2022

dwijnand self-assigned this Dec 12, 2022

dwijnand approved these changes Dec 12, 2022

View reviewed changes

odersky merged commit 6383025 into scala:main Dec 14, 2022

odersky deleted the fix-16368 branch December 14, 2022 19:51

Kordyjan modified the milestones: 3.3.0-RC1, 3.0.0, 3.3.0 Aug 2, 2023

abrighton mentioned this pull request Aug 4, 2023

Possible issue with scala-3: dotty.tools.FatalError: cannot resolve reference sirthias/borer#661

Closed

This was referenced Aug 20, 2023

Scala 3.3.x support blocked by potential bug in Scala 3.3.0 ReactiveMongo/ReactiveMongo#1250

Open

Scala 3.3.0 compilation error SamTheisens/ReactiveMongo#2

Draft

SamTheisens mentioned this pull request Aug 20, 2023

Feature/scala 3.3.x ReactiveMongo/ReactiveMongo#1251

Closed

3 tasks

SamTheisens mentioned this pull request Sep 15, 2023

PR #16373 introduces a "cannot resolve reference to type" compilation error on ReactiveMongo #18555

Closed

mdedetrich mentioned this pull request Sep 15, 2023

Publish pekko based ReactiveMongo binaries ReactiveMongo/ReactiveMongo#1245

Merged

3 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Make it a fatal error if erasure cannot resolve a type #16373

Make it a fatal error if erasure cannot resolve a type #16373

odersky commented Nov 18, 2022

jchyb commented Nov 18, 2022

odersky commented Nov 18, 2022

jchyb commented Nov 21, 2022

odersky commented Nov 21, 2022

jchyb commented Nov 28, 2022

odersky commented Dec 7, 2022

odersky commented Dec 9, 2022

odersky commented Dec 9, 2022

odersky commented Dec 9, 2022

mdedetrich commented Sep 15, 2023

Make it a fatal error if erasure cannot resolve a type #16373

Make it a fatal error if erasure cannot resolve a type #16373

Conversation

odersky commented Nov 18, 2022

jchyb commented Nov 18, 2022

odersky commented Nov 18, 2022

jchyb commented Nov 21, 2022

odersky commented Nov 21, 2022

jchyb commented Nov 28, 2022

odersky commented Dec 7, 2022

odersky commented Dec 9, 2022

odersky commented Dec 9, 2022

odersky commented Dec 9, 2022

mdedetrich commented Sep 15, 2023