Zinc reports incorrect class file names #1233
Assignees
Comments
|
When compiling the same project using Scala 3.3.0, |
|
In /**
* Registers only non-local generated classes in the callback by extracting
* information about its names and using the names to generate class file paths.
*
* Mimics the previous logic that was present in `Analyzer`, despite the fact
* that now we construct the names that the compiler will give to every non-local
* class independently of genbcode.
*
* Why do we do this? The motivation is that we want to run the incremental algorithm
* independently of the compiler pipeline. This independence enables us to:
*
* 1. Offload the incremental compiler logic out of the primary pipeline and
* run the incremental phases concurrently.
* 2. Know before the compilation is completed whether another compilation will or
* will not be required. This is important to make incremental compilation work
* with pipelining and enables further optimizations; for example, we can start
* subsequent incremental compilations before (!) the initial compilation is done.
* This can buy us ~30-40% faster incremental compiler iterations.
*
* This method only takes care of non-local classes because local classes have no
* relevance in the correctness of the algorithm and can be registered after genbcode.
* Local classes are only used to construct the relations of products and to produce
* the list of generated files + stamps, but names referring to local classes **never**
* show up in the name hashes of classes' APIs, hence never considered for name hashing.
*
* As local class files are owned by other classes that change whenever they change,
* we could most likely live without adding their class files to the products relation
* and registering their stamps. However, to be on the safe side, we will continue to
* register the local products in `Analyzer`.
*
* @param allClassSymbols The class symbols found in all the compilation units.
*/
def registerGeneratedClasses(classSymbols: Iterator[Symbol]): Unit = ... |
|
In /**
* Replicate the behaviour of `fullName` with a few changes to the code to produce
* correct file-system compatible full names for non-local classes. It mimics the
* paths of the class files produced by genbcode.
*
* Changes compared to the normal version in the compiler:
*
* 1. It will use the encoded name instead of the normal n2. It will not skip the name of the package object class (required for the class file path).
*
* Note that using `javaBinaryName` is not useful for these symbols because we
* need the encoded names. Zinc keeps track of encoded names in both the binary
* names and the Zinc names.
*
* @param symbol The symbol for which we extract the full name.
* @param separator The separator that we will apply between every name.
* @param suffix The suffix to add at the end (in case it's a module).
* @param includePackageObjectClassNames Include package object class names or not.
* @return The full name.
*/
override def fullName(
symbol: Symbol,
separator: Char,
suffix: CharSequence,
includePackageObjectClassNames: Boolean
): String = ... |
|
Thanks for everyone looking into this. Does anyone know if there is going to be a second attempt at approaching this issue? |
|
Dale and Lukas were working on it today, so they'll probably have something before too much longer. |
|
Yep, working on it. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Steps
Warning: The compilation can take 35+ seconds to complete. Check the note at the end of this post.
The output should look similar to the following, modulo local machine paths. Only the names of the
.classfiles are important.Problem
These names would never be generated by the
scalaccompiler, because they are too long for the local file systems and can cause exceptions when opened for reading in a JVM process. They exceed 255 characters in length.cd target/scala-2.13/classes lsThese are the actual
.classfiles generated byscalac.These names come from https://github.com/scala/scala/blob/98a42345385cc8adccde36c21f3192d4b4862087/src/reflect/scala/reflect/internal/StdNames.scala#L49-L86.
Expectation
The
compileOutputsreported by Zinc should match the actual class files generated byscalac.In IntelliJ IDEA, this problem manifests as part of our compilation pipeline which uses Zinc.
https://youtrack.jetbrains.com/issue/SCL-21477/File-name-too-long-error-when-building-projects-like-Apache-Spark
This is a pretty serious issue. I would appreciate any help in diagnosing it.
Notes
On a side note, the compilation of the single Scala source file
main.scalatakes 35+ seconds and cannot be canceled withctrl+C. This seems excessive, since most of the definitions are empty object definitions.The text was updated successfully, but these errors were encountered: