Regression on compiling Scala 3: "Can't find source file for a.A.Inner_sel"

[eed3si9n]

steps

See scala/scala3#20157

problem

The following scripted tests failed on Scala 3 repo:

Error:  (sbt-test / scripted) Failed tests:
Error:  	pipelining/Yjava-tasty-fromjavaobject
Error:  	pipelining/Yjava-tasty-paths
Error:  	sbt-dotty/dotty-knowledge.i17
Error:  	source-dependencies/false-error
Error:  	source-dependencies/inline-rec-mut

[info] [error] ## Exception when compiling 3 sources to /tmp/sbt_5c1f43c7/a-enum-classes
[info] [error] java.lang.RuntimeException: Can't find source file for a.A.Inner_sel
[info] [error] scala.sys.package$.error(package.scala:30)
[info] [error] sbt.internal.inc.Analysis$.$anonfun$sources$1(Analysis.scala:105)
[info] [error] scala.Option.getOrElse(Option.scala:189)
[info] [error] sbt.internal.inc.Analysis$.sourceFileForClass$1(Analysis.scala:105)
[info] [error] sbt.internal.inc.Analysis$.isJavaClass$1(Analysis.scala:107)
[info] [error] sbt.internal.inc.Analysis$.$anonfun$sources$2(Analysis.scala:108)
[info] [error] sbt.internal.inc.Analysis$.$anonfun$sources$2$adapted(Analysis.scala:108)
[info] [error] scala.collection.TraversableLike.$anonfun$partition$1(TraversableLike.scala:450)
[info] [error] scala.collection.immutable.HashMap$HashMapKeys.$anonfun$foreach$1(HashMap.scala:150)
[info] [error] scala.collection.immutable.HashMap$HashMap1.foreachEntry(HashMap.scala:401)
[info] [error] scala.collection.immutable.HashMap$HashTrieMap.foreachEntry(HashMap.scala:735)
[info] [error] scala.collection.immutable.HashMap$HashTrieMap.foreachEntry(HashMap.scala:735)
[info] [error] scala.collection.immutable.HashMap$HashMapKeys.foreach(HashMap.scala:150)
[info] [error] scala.collection.TraversableLike.partition(TraversableLike.scala:450)
[info] [error] scala.collection.TraversableLike.partition$(TraversableLike.scala:448)
[info] [error] scala.collection.AbstractTraversable.partition(Traversable.scala:108)
[info] [error] sbt.internal.inc.Analysis$.sources(Analysis.scala:108)
[info] [error] sbt.internal.inc.IncrementalCommon$CycleState$IncrementalCallbackImpl.mergeAndInvalidate(IncrementalCommon.scala:193)
[info] [error] sbt.internal.inc.IncrementalCommon$CycleState$IncrementalCallbackImpl.completeCycle(IncrementalCommon.scala:209)
[info] [error] sbt.internal.inc.AnalysisCallback.getCycleResultOnce(Incremental.scala:940)
[info] [error] sbt.internal.inc.Incremental$$anon$2.run(Incremental.scala:455)
[info] [error] sbt.internal.inc.IncrementalCommon$CycleState.next(IncrementalCommon.scala:117)
[info] [error] sbt.internal.inc.IncrementalCommon$$anon$1.next(IncrementalCommon.scala:56)
[info] [error] sbt.internal.inc.IncrementalCommon$$anon$1.next(IncrementalCommon.scala:52)
[info] [error] sbt.internal.inc.IncrementalCommon.cycle(IncrementalCommon.scala:265)
[info] [error] sbt.internal.inc.Incremental$.$anonfun$incrementalCompile$8(Incremental.scala:409)
[info] [error] sbt.internal.inc.Incremental$.withClassfileManager(Incremental.scala:496)
[info] [error] sbt.internal.inc.Incremental$.incrementalCompile(Incremental.scala:396)
[info] [error] sbt.internal.inc.Incremental$.apply(Incremental.scala:170)
[info] [error] sbt.internal.inc.IncrementalCompilerImpl.compileInternal(IncrementalCompilerImpl.scala:534)
[info] [error] sbt.internal.inc.IncrementalCompilerImpl.$anonfun$compileIncrementally$1(IncrementalCompilerImpl.scala:488)
[info] [error] sbt.internal.inc.IncrementalCompilerImpl.handleCompilationError(IncrementalCompilerImpl.scala:332)
[info] [error] sbt.internal.inc.IncrementalCompilerImpl.compileIncrementally(IncrementalCompilerImpl.scala:425)
[info] [error] sbt.internal.inc.IncrementalCompilerImpl.compile(IncrementalCompilerImpl.scala:137)
[info] [error] sbt.Defaults$.compileIncrementalTaskImpl(Defaults.scala:2413)
[info] [error] sbt.Defaults$.$anonfun$compileIncrementalTask$2(Defaults.scala:[236](https://github.com/scala/scala3/actions/runs/8690208123/job/23829593105#step:9:237)3)
[info] [error] sbt.internal.server.BspCompileTask$.$anonfun$compute$1(BspCompileTask.scala:41)
[info] [error] sbt.internal.io.Retry$.apply(Retry.scala:47)
[info] [error] sbt.internal.io.Retry$.apply(Retry.scala:29)
[info] [error] sbt.internal.io.Retry$.apply(Retry.scala:24)
[info] [error] sbt.internal.server.BspCompileTask$.compute(BspCompileTask.scala:41)
[info] [error] sbt.Defaults$.$anonfun$compileIncrementalTask$1(Defaults.scala:2361)
[info] [error] scala.Function1.$anonfun$compose$1(Function1.scala:49)
[info] [error] sbt.internal.util.$tilde$greater.$anonfun$$u2219$1(TypeFunctions.scala:63)
[info] [error] sbt.std.Transform$$anon$4.work(Transform.scala:69)
[info] [error] sbt.Execute.$anonfun$submit$2(Execute.scala:283)
[info] [error] sbt.internal.util.ErrorHandling$.wideConvert(ErrorHandling.scala:24)
[info] [error] sbt.Execute.work(Execute.scala:292)
[info] [error] sbt.Execute.$anonfun$submit$1(Execute.scala:283)
[info] [error] sbt.ConcurrentRestrictions$$anon$4.$anonfun$submitValid$1(ConcurrentRestrictions.scala:265)
[info] [error] sbt.CompletionService$$anon$2.call(CompletionService.scala:65)
[info] [error] java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
[info] [error] java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
[info] [error] java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
[info] [error] java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
[info] [error] java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
[info] [error] java.base/java.lang.Thread.run(Thread.java:833)

[eed3si9n]

I guess more than anything these are reflection of the lack of test coverage against the Scala compiler bridges that seceded away from Zinc repo.

[eed3si9n]

@bishabosha wrote on scala/scala3#20157 (comment)

@eed3si9n on the current main branch with sbt 1.9.9, I printed out the relations after a/compile in that task and it seems some classes are missing (such as a.A.Inner_sel):

class names: Relation [
  ${BASE}/a/src/main/scala/a/A.java -> a.A
  ${BASE}/a/src/main/scala/a/A.java -> a.A$.Inner
  ${BASE}/a/src/main/scala/a/A.java -> a.A$.Inner_sel
  ${BASE}/a/src/main/scala/a/AImport.java -> a.AImport
  ${BASE}/a/src/main/scala/a/AImport.java -> a.AImport.Inner
]

here are the set of apis.allInternalClasses:

[
  a.A,
  a.AImport.Inner,
  a.A$.Inner_sel,
  a.A.Inner_sel,
  a.AImport,
  a.A$.Inner,
  a.A.Inner,
  a.AImport$.Inner
]

I'm still not sure where the source of regression is, but one thing we can try is make isJavaClass more robust -

zinc/internal/zinc-core/src/main/scala/sbt/internal/inc/Analysis.scala

Lines 103 to 107 in 3de8557

	def sourceFileForClass(className: String): VirtualFileRef =
	a.relations.definesClass(className).headOption.getOrElse {
	sys.error(s"Can't find source file for $className")
	}
	def isJavaClass(className: String) = sourceFileForClass(className).id.endsWith(".java")

and try a.A$.Inner_sel when a.A.Inner_sel if not found by substituting the last . with $..

[eed3si9n]

With scala/scala3#20279, pipelining/Yjava-tasty-fromjavaobject works on my machine with sbt 1.10.0-RC2.