Signature files produced by `-Ypickle-write` do not contain enclosing filename

[olafurpg]

Currently, *.sig files produced by -Ypickle-write do not contain the filename where the symbols originate. The filename is normally included in *.class files and is used by IDEs to navigate to library dependencies.

See https://youtrack.jetbrains.com/issue/SCL-16543 for a related discussion on making IntelliJ Scala pick up *.sig files.

Would it be possible somehow to encode the filename in *.sig files? I'm not sure exactly where would be the place for such information 🤔

[SethTisue]

@lrytz who on the team is best positioned to comment on this?

[lrytz]

Ping @retronym - is there a way to do it without extending the pickle format, so that an existing 2.13.1 compiler could read a pickle generated by 2.13.x that contains the source file name?

[retronym]

One solution would be to wrap the Pickle in a mostly-empty .class file. That could contain any classfile attributes.

I considered this approach at one stage as it is nice for tooling, but in the end I opted for the more efficient direct representation of just putting the pickle bytes in the .sig file directly.

Without extending the pickle format, we might be able to encode the filename attribute as a synthetic annotation inside the pickle.

[dwijnand]

The team at Jetbrains just implemented a guess: https://github.com/JetBrains/intellij-scala/blob/d90c4508cb0a2db7abaf1f284ef80b5e5ea889ef/scala/decompiler/src/org/jetbrains/plugins/scala/decompiler/Decompiler.scala#L52.

[szeiger]

Has there been any further work towards generating class files? This feature would also be useful for pipelined compilation of downstream Java projects (either pure Java or mixed Scala/Java). sbt can split up mixed projects internally and wait until the real class files of dependencies are available before compiling the Java sources (instead of simply calling Zinc's compileMixed). This is probably good enough for most use cases as Java compilation tends to be fast and mixed projects usually contain very little Java. But it doesn't work for integrating pipelined compilation into Bazel. Even the simplest use case (using outlines produced on the fly by the compiler to remove the extra ijar action that has to run sequentially) is impossible if we don't get full class files that can be used by javac.