Introduce Labeled blocks, and use them in PatternMatcher #4982
Conversation
There was a problem hiding this comment.
Hello, and thank you for opening this PR! 🎉
All contributors have signed the CLA, thank you! ❤️
Commit Messages
We want to keep history, but for that to actually be useful we have
some rules on how to format our commit messages (relevant xkcd).
Please stick to these guidelines for commit messages:
- Separate subject from body with a blank line
- When fixing an issue, start your commit message with
Fix #<ISSUE-NBR>:- Limit the subject line to 72 characters
- Capitalize the subject line
- Do not end the subject line with a period
- Use the imperative mood in the subject line ("Add" instead of "Added")
- Wrap the body at 80 characters
- Use the body to explain what and why vs. how
adapted from https://chris.beams.io/posts/git-commit
Have an awesome day! ☀️
|
Can anyone tell me how I can get the CI to take into account the version of scala-backend I have pushed at https://github.com/dotty-staging/scala/tree/dotty-labeled? |
|
Push it to a branch on http://github.com/lampepfl/scala or replace lampepfl by dotty-staging in .gitmodules under the scala-backend section. |
| final val OBJECTDEF = 175 | ||
|
|
||
| // In binary: 101100EI | ||
| final val LABELED = 144 |
There was a problem hiding this comment.
Need to bump the tasty MajorVersion in this file.
There was a problem hiding this comment.
I don't think so. I think the last bump to version 10 happened after the latest release.
There was a problem hiding this comment.
We should always bump when we break tasty. If we merge and you don't bump, people will start getting confusing error messages in their build when incremental compilation mixes old and new tasty files.
| changePrec(GlobalPrec) { keywordStr("return") ~ optText(expr)(" " ~ _) } | ||
| val sym = from.symbol | ||
| if (sym.is(Label)) | ||
| changePrec(GlobalPrec) { keywordStr("return@") ~ toText(sym.name) ~ optText(expr)(" " ~ _) } |
There was a problem hiding this comment.
Very minor bikeshed but I'm not a fan of using @ since it looks too much like an annotation. Maybe write return[foo] instead ?
Thanks. That worked :) |
sjrd
force-pushed
the
labeled
branch
2 times, most recently
from
9c19830
to
432aaaa
Compare
sjrd
changed the title
|
Good to review :-D Who feels like it? |
|
What's missing to completely remove labeldefs? |
|
test performance please |
|
Need to update the |
| @@ -426,6 +426,9 @@ class TreePickler(pickler: TastyPickler) { | |||
| case CaseDef(pat, guard, rhs) => | |||
| writeByte(CASEDEF) | |||
| withLength { pickleTree(pat); pickleTree(rhs); pickleTreeUnlessEmpty(guard) } | |||
| case Labeled(bind, expr) => | |||
| writeByte(LABELED) | |||
| withLength { pickleTree(bind); pickleTree(expr) } | |||
There was a problem hiding this comment.
Right now we will never pickle Labeled trees as they are generated in pattern matcher. We should add this code only when we will need it for while and do while
tailrec-generated labels, and |
|
@liufengyun any idea why "test performance please" above didn't work? |
|
test performance please @smarter the polling service ended, just restarted. |
|
performance test scheduled: 1 job(s) in queue, 1 running. |
|
It'd be interesting to check if https://gist.github.com/smarter/1c564e65514d328462e7453162ccfb65 which works in scalac (but was removed from the test suite for being too slow) now works with this PR (it stackoverflows on master). |
nicolasstucki
changed the title
|
It "works" as far as the compiler itself is concerned, but it produces too much code for the JVM: |
|
At least we do not crash before reaching that limit. |
|
@sjrd interesting, I suppose that's something that would be fixed with more optimizations in the pattern matcher? |
|
Yes and no. One thing would be to short-circuit Other than that, there's not much room for optimization of this particular pattern-match. It is specifically designed to prevent any kind of optimization: the bits are always alternating at the top-level, so the optimizer can never collapse two consecutive cases. It has to redo all the tests from the top every time. scalac does not fare better than my PR, here. |
|
Just for curiosity, if you reorder the cases to not alternate does it optimize and compile? |
|
Yes, it does. The tree after patmat is https://gist.github.com/sjrd/803ddf59e4d17a5ffa445e3f3c16dfd4 (around 5 kLoC, versus 17 kLoC for the original, failing one). |
| "elimRedundantTests" -> elimRedundantTests, | ||
| "inlineLabelled" -> inlineLabelled, | ||
| "mergeVars" -> mergeVars, | ||
| "mergeTests" -> mergeTests, |
There was a problem hiding this comment.
Could the removed optimizations be brought back later? Or are they subsumed by mergeTests ?
There was a problem hiding this comment.
They are indeed subsumed by mergeTests, combined with the inherent simpler nature of Labeled versus label-defs (e.g., hoistLabels is totally unnecessary with Labeled).
|
Performance test finished successfully: Visit http://dotty-bench.epfl.ch/4982/ to see the changes. Benchmarks is based on merging with master (6ee3c62) |
|
Would be good to verify that this PR fixes #4563 |
Done.
It does. I added a test case. |
tests/pos/i4563.scala
Outdated
| @@ -0,0 +1,12 @@ | |||
| case class Foo(str: String) | |||
There was a problem hiding this comment.
This test already compiles and runs on master. TBH, I am not sure this minimises the original issue.
Anyway, this was reported as a runtime issue, so this should be a run test
tests/pos/i4563.scala
Outdated
| @@ -0,0 +1,12 @@ | |||
| case class Foo(str: String) | |||
There was a problem hiding this comment.
This is missing the extends AnyVal
Done. I think I've addressed all the comments that have been raised so far. |
|
Assigning to @odersky since he wrote most of the current PatternMatcher |
|
This PR also fixes #1313. Test case to be added when it's not 1am: object HelloWorld {
def main(args: Array[String]): Unit = println("hello world")
def test(x: Any, y: Int): Int = {
x match {
case List(1, 2, 3) => 1
case List(3, 4, 5) => 2
case 6 if y > 5 => 3
case 6 => 4
case _ => 5
}
}
def test2(x: Int, y: Int): Int = {
(x: @scala.annotation.switch) match {
case 6 if y > 5 => 1
case 6 => 2
case 12 => 3
case 14 => 4
case _ => 5
}
}
}No "switch" warning! Tree after the pattern matcher: package <empty> {
final lazy module val HelloWorld: HelloWorld$ = new HelloWorld$()
@scala.annotation.internal.SourceFile("tests\\pos\\HelloWorld.scala")
final module
class HelloWorld$() extends Object() {
def main(args: Array[String]): Unit = println("hello world")
def test(x: Any, y: Int): Int =
{
matchResult1[Int]:
{
case val x1: Any(x) = x
if x1.$isInstanceOf$[List[Any]] then
{
case val x8: List[Any] = x1.asInstanceOf[List[Any]]
case val x9: Some[List[Any]] = List.unapplySeq[Any](x8)
if x9.isEmpty.unary_! then
{
case val x10: List[Any] = x9.get
if x10.lengthCompare(3).==(0) then
{
case val x11: Any = x10.apply(0)
case val x12: Any = x10.apply(1)
case val x13: Any = x10.apply(2)
if 1.==(x11).&&(2.==(x12)).&&(3.==(x13)) then
return[matchResult1] 1
else ()
if 3.==(x11).&&(4.==(x12)).&&(5.==(x13)) then
return[matchResult1] 2
else ()
}
else ()
}
else ()
}
else ()
if 6.==(x1) then
{
if y.>(5) then return[matchResult1] 3 else ()
return[matchResult1] 4
}
else ()
return[matchResult1] 5
return[matchResult1] throw new MatchError(x1)
}
}
def test2(x: Int, y: Int): Int =
{
matchResult2[Int]:
{
case val x14: Int(x) @switch = x: Int(x) @switch
x14 match
{
case 6 =>
if y.>(5) then return[matchResult2] 1 else ()
return[matchResult2] 2
case 12 => return[matchResult2] 3
case 14 => return[matchResult2] 4
case _ =>
return[matchResult2] 5
return[matchResult2] throw new MatchError(x14)
}
}
}
}
} |
This is consistent with those produced by the current pattern matcher.
| val PatMatMatchFailName = new UniqueNameKind("matchFail") | ||
| val PatMatSelectorName = new UniqueNameKind("selector") | ||
| val PatMatAltsName = new UniqueNameKind("matchAlts") | ||
| val PatMatResultName = new UniqueNameKind("matchResult") |
|
Good to merge unless @sjrd has something to add. |
|
Thank you :) I don't have anything to add. Changes to loops and tailrec are best left as separate PRs. Let's ship it! |
|
Oh, I'm just going to update the submodule commit reference to the new HEAD of |
A Labeled block is an expression tree of the form
label[T]: {
expr
}
where `expr` must conform to the type `T`. In addition, within
`expr` (but nowhere else), return from the label is allowed:
return[label] e
where `e` must conform to the type `T` as well.
If execution of `expr` completes normally (rather than throwing an
exception or returning, etc.), then the result of evaluating the
`Labeled` block is the result of `expr`. If a `return[label] e` is
reached, the execution of `expr` is interrupted, and the result of
evaluating the `Labeled` block is the result of evaluating the
argument `e`.
Implementation-wise, a `Labeled` block is represented as a `Tree`
with the shape:
Labeled(Bind(labelName), expr)
where the `Bind` nodes holds the definition of the label symbol.
That symbol is a term symbol with the flag `Label` (but not
`Method`, unlike symbols for label-defs) and whose `info` is the
result type `T` of the labeled block.
We use those new `Labeled` blocks in `PatternMatcher`, instead of
label-defs. This is the first step towards completely removing
label-defs from the compiler.
This commit structurally fixes a few issues:
* It fixes scala#1313 through the `mergeTests` optimization.
* It fixes scala#4563 because Labeled blocks are erasure-friendly.
* It does a big step towards fixing the upstream test t10387: the
compiler can get to the back-end on that test, but it produces
too much bytecode for a single JVM method. We do add a sister
test t10387b which works because optimizations can kick in.
A Labeled block is an expression tree of the form
where
exprmust conform to the typeT. In addition, withinexpr(but nowhere else), return from the label is allowed:return[label] ewhere
emust conform to the typeTas well.If execution of
exprcompletes normally (rather than throwing an exception or returning, etc.), then the result of evaluating theLabeledblock is the result ofexpr. If areturn[label] eis reached, the execution ofexpris interrupted, and the result of evaluating theLabeledblock is the result of evaluating the argumente.Implementation-wise, a
Labeledblock is represented as aTreewith the shape:where the
Bindnodes holds the definition of the label symbol. That symbol is a term symbol with the flagLabel(but notMethod, unlike symbols for label-defs) and whoseinfois the result typeTof the labeled block.We use those new
Labeledblocks inPatternMatcher, instead of label-defs. This is the first step towards completely removing label-defs from the compiler.This commit structurally fixes a few issues:
mergeTestsoptimization.