First step to pipelining support - enable reading Java symbols from TASTy #19074
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
bishabosha
added
the
needs-minor-release
|
@nicolasstucki this PR contains some changes to handling of TASTy attributes so I request your review here |
|
one open question here is not requiring any flag by default to read |
sjrd
reviewed
Nov 27, 2023
sbt-test/pipelining/Yjava-tasty-result-types/a/src/main/scala/a/A.java
Outdated
Show resolved
Hide resolved
bishabosha
force-pushed
the
topic/java-tasty
branch
from
f6e2f43
to
9792f13
Compare
nicolasstucki
added a commit
that referenced
this pull request
Nov 27, 2023
We generalize the internal encoding of `Attributes` to be a list of tags. Then we add add helper methods to have simpler ways to interact with this abstraction using booleans. This implies that the pickling/unpickling can be agnostic of the semantics of each tag. Therefore reducing the number of places that need to be updated when we add a new tag. Useful for #19074, #19033, and #18948.
nicolasstucki
added a commit
to dotty-staging/dotty
that referenced
this pull request
Nov 27, 2023
We generalize the internal encoding of `Attributes` to be a list of tags. Then we add add helper methods to have simpler ways to interact with this abstraction using booleans. This implies that the pickling/unpickling can be agnostic of the semantics of each tag. Therefore reducing the number of places that need to be updated when we add a new tag. Useful for scala#19074, scala#19033, and scala#18948.
bishabosha
force-pushed
the
topic/java-tasty
branch
2 times, most recently
from
2994ebb
to
7913851
Compare
sjrd
approved these changes
Nov 27, 2023
bishabosha
force-pushed
the
topic/java-tasty
branch
from
7913851
to
fef4875
Compare
nicolasstucki
approved these changes
Nov 28, 2023
- Rename JavaEnumTrait flags to JavaEnum (the actual flags set) - test java enum in SealedDescendantsTest
- Keep Java compilation units up to Pickler phase if -Yjava-tasty. Skip phases for Java when not needed. - Add JAVAattr and OUTLINEattr TASTy attributes, ELIDED tree tag. ELIDED trees are pickled as rhs of java term definitions. ELIDED trees can only be unpickled if OUTLINEattr is present. For now OUTLINEattr implies JAVAattr. In the future we might expand OUTLINEattr to include outline Scala typing. - write java tasty files to a special jar, set with -Yjava-tasty-output this option is for testing purposes only.
bishabosha
force-pushed
the
topic/java-tasty
branch
from
fef4875
to
9b12e4a
Compare
nicolasstucki
approved these changes
Nov 28, 2023
bishabosha
added a commit
that referenced
this pull request
Apr 4, 2024
This includes support for a single pass pipelined build, compatible with sbt's `ThisBuild/usePipelining`, - adds `-Ypickle-java` and `-Ypickle-write` flags, expected by Zinc when pipelining is enabled in sbt. - when `-Ypickle-write <directory|jar>` is set, then write tasty from pickler to that output, (building upon #19074 support for Java signatures in TASTy files). - call `apiPhaseCompleted` and `dependencyPhaseCompleted` callbacks, which will activate early downstream compilation - calls `generatedNonLocalClass` callbacks early, which enables Zinc to run the incremental algorithm before starting downstream compilation (including checking for macro definitions). generally this can be reviewed commit-by-commit, as they each do an isolated feature. As well as many tests in the `sbt-test/pipelining` directory, this has also been tested locally on `akka/akka-http`, `apache/incubator-pekko`, `lichess-org/lila`, `scalacenter/scaladex`, `typelevel/fs2`, `typelevel/http4s`, `typelevel/cats`, `slick/slick`. This PR sets the ground work for an optional 2-pass compile (reusing the `OUTLINEattr`), which should use a faster frontend (skipping rhs when possible) before producing tasty signatures fixes #19743
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Release Notes
Road to Pipelined Builds
We made the next concrete preparation for introducing pipelined Scala 3 builds. Now TASTy can store outline signatures, and additionally the signatures of Java source files. This is the only TASTy breaking change required to introduce pipelining, which means that once it is ready, pipelined build support will be able to release in an upcoming patch version of Scala Next.
OUTLINEattrTASTy attribute) only store what is necessary for separate compilation (i.e. method bodies can be elided). This will enable in the future the possibility of a faster type checking phase because a lot of work is no longer necessary to produce this outline TASTy. Elided expressions are represented by the newELIDEDtree in TASTy.JAVAattrTASTy attribute) can be produced faster than waiting for class files fromjavac, which will be necessary to enable pipelined builds that include Java sources.Full Summary
This PR contains the minimal forward incompatible changes of pipelining - reading Java and outline symbols from TASTy.
The other changes to implement pipelining have no impact on TASTy or the standard library - so can come in a patch release.
To test reading TASTy produced from Java defined classes, we add two private flags
-Yjava-tastyand-Yjava-tasty-output, which are not expected to be used by any build tool. The tests allow us to write just the java signatures to TASTy, package them in a jar, and then read the Java TASTy from the classpath.-Yjava-tastyis set. Skip phases for Java when not needed.JAVAattrandOUTLINEattrTASTy attributes,ELIDEDtree tag.ELIDEDtrees are pickled as rhs of java term definitions.ELIDEDtrees can only be unpickled ifOUTLINEattris present.JAVAattrTASTy attribute. As currently we outline parse Java files we also set theOUTLINEattr.OUTLINEattrto include outline Scala typing.OUTLINEattrandJAVAattrdo not need any special flags to read from the classpath, however to read outline tasty in the-from-tastymode (sotasty-inspectorandscaladocare included) we do require an explicit-Yallow-outline-from-tasty, as method bodies are required for full functionality.-Yjava-tasty-outputthis option is for testing purposes only.
Fix sealedDescendants method for Java Enums.
JavaEnumTraitflags toJavaEnum(reflecting the actual flags set)SealedDescendantsTestfixes #15908