-
Notifications
You must be signed in to change notification settings - Fork 3.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Stabilize receiver of extension method application so that implicits accessible via the prefix can be candidate implicit arguments #5999
Conversation
Minor admin note: could you please retitle the commit/PR with a description of the change, rather than the ticket reference? We've been leaving the ticket reference as a |
def stabilizeQual(tree: Tree): Option[(Tree, ValDef)] = tree match { | ||
case Select(qual, name) if qual.tpe.isStable => None | ||
case Select(qual, name) => | ||
val vsym = context.owner.newValue(unit.freshTermName(), context.owner.pos, SYNTHETIC) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think the qual.pos.focus
will be more precise than context.owner.pos
for the synthetic tree.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks.
val newQual = Ident(vsym).setType(qual.tpe) | ||
val newSelect = makeAccessible(treeCopy.Select(tree, newQual, name), tree.symbol, singleType(NoPrefix, vsym), newQual)._1 | ||
if(newSelect.tpe.contains(vsym)) | ||
Some((newSelect, vdef)) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't see any atPos
or typedPos
that positions the synthetic trees, but I guess we can discuss the required positioning cleanups after we discuss the actual functional change.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ditto ...
case _ => None | ||
} | ||
|
||
stabilizeQual(tree).map { case (newLhs, vdef) => |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I wonder if we can use mt.isDependentMethodType
to drive whether the old or new logic is used. I still don't have a clear picture of the problem or solution, so just brainstorming a little...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the pointer ... I'll investigate.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That won't help here. The method type contains the prefix, but it's not a dependent method in the sense which isDependentMethodType
is testing for.
test/files/pos/t4225.scala
Outdated
} | ||
} | ||
|
||
implicit class Ops[F <: Foo](val f0: F) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
A test variant with a by-name implicit f0
here might show up a problem with evaluation order by hoisting out the val. We deal with the same sort of issue in the named/default args transform, we would need to share some of that implementation in this case.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I've added some test variants which cover by name arguments (for the ops class and for the ops methods) and (by name) right associative ops and they don't cause any problems. Do you have a particular scenario in mind which you think might be problematic?
} | ||
case _ => None | ||
} | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could you add some comment "rewrites expr
to fxpr
because ..., see scala/bug#X"? It will save time for people driving by here in the future.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Will do ...
996c771
to
b392aa7
Compare
@retronym PR title and description tweaked as requested. |
@milessabin Forgive me if this is a stupid question, but is it possible to implement the same mechanism of synthetic val to resolve milessabin/shapeless#738 with |
The commit messages are still empty. Please start with a prescriptive/imperative title (to keep it short, don't mention tickets here), then a body with the description. End with references to tickets. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The example here scala/bug#4225 (comment) still doesn't compile
class Foo {
class Bar
object Bar {
implicit def fromString(a: String) = new Bar
}
def andThen(b : Bar) = b
}
object Test {
(new Foo) andThen ("Bar")
}
gives
Test.scala:10: error: type mismatch;
found : String("Bar")
required: _1.Bar where val _1: Foo
(new Foo) andThen ("Bar")
^
one error found
@@ -880,8 +880,50 @@ trait Typers extends Adaptations with Tags with TypersTracking with PatternTyper | |||
} | |||
} | |||
) | |||
else | |||
typer1.typed(typer1.applyImplicitArgs(tree), mode, pt) | |||
else { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is the else
expr of if (original != EmptyTree && pt != WildcardType)
. The if
expr is not obvious to me - do you understand what it does and why we need the two branches? And why / if your fix is not needed in the if
case?
else | ||
typer1.typed(typer1.applyImplicitArgs(tree), mode, pt) | ||
else { | ||
if(tree.symbol != null && !tree.symbol.isConstructor) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I guess you observed the null
case, maybe under erroneous trees? A comment would help.
typer1.typed(typer1.applyImplicitArgs(tree), mode, pt) | ||
else { | ||
if(tree.symbol != null && !tree.symbol.isConstructor) { | ||
def stabilizeQual(tree: Tree): Option[(Tree, ValDef)] = tree match { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could the transformation be done using this?
case treeInfo.Applied(fun @ Select(qual, name), targs, argss) => ...
I also wonder if assigning the types manually to the Select / Apply / TypeApply nodes is necessary. Could we just build the trees and run the typer? Probably the new synthetic value would have to be entered in the typer's scope. Also, the select's symbol might need to be set manually to prevent going into overloading resolution. Similar case for named/default args: https://github.com/scala/scala/blob/v2.13.0-M2/src/compiler/scala/tools/nsc/typechecker/NamesDefaults.scala#L173-L189. See also #558.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If we have the types available to assign why would we run the typer? Wouldn't that be more expensive?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It seems to me that inlining pieces of typedSelect
and typedApply
makes the code unnecessarily long / harder to maintain. I don't think we have a performance issue here, we'd just run through a few methods of the typer.
case _ => None | ||
} | ||
|
||
// Rewrites "qual.name ..." to "{ val lhs = qual ; lhs.name ... }" in cases where |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Add a reference to "scala/bug#5946"
val newQual = Ident(vsym).setType(qual.tpe) | ||
val newSelect0 = atPos(tree.pos)(treeCopy.Select(tree, newQual, name)) | ||
val newSelect = makeAccessible(newSelect0, tree.symbol, singleType(NoPrefix, vsym), newQual)._1 | ||
if(newSelect.tpe.contains(vsym)) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
So this is to identify whether the transformation is necessary, right? Do you see a way to check this beforehand?
b392aa7
to
b3b2b4c
Compare
@lrytz you're right that this didn't in fact fix SI-4225, although it did fix some related issues. I've now completely reworked the fix and it now solves both the issues this PR originally addressed and SI-4225 proper. The earlier commits are obsolete ... I should probably just squash this. I think the implementation is now a lot simpler ... the problematic cases are detected directly in |
The broadened scope of this fix means that one existing |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nice, this is definitely much nicer than the previous solution!
val vsym = insertionContext.owner.newValue(unit.freshTermName(nme.STABILIZER_PREFIX), qual.pos.focus, SYNTHETIC | ARTIFACT | STABLE) | ||
vsym.setInfo(qual.tpe) | ||
insertionContext.scope enter vsym | ||
val vdef = ValDef(vsym, qual) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we need a changeOwner
here, for symbols defined within the qual
, similar to named / default arguments: https://github.com/scala/scala/blob/v2.13.0-M2/src/compiler/scala/tools/nsc/typechecker/NamesDefaults.scala#L173-L189
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Gotcha.
vsym.setInfo(qual.tpe) | ||
insertionContext.scope enter vsym | ||
val vdef = ValDef(vsym, qual) | ||
vdef.pos.makeTransparent |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It seems to me that this just computes a new Position
value and discards it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hmm ... you're right.
|
||
if(insertionContext != NoContext) { | ||
val vsym = insertionContext.owner.newValue(unit.freshTermName(nme.STABILIZER_PREFIX), qual.pos.focus, SYNTHETIC | ARTIFACT | STABLE) | ||
vsym.setInfo(qual.tpe) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we might have the same issue as scala/bug#7694 here, fixed in named / default args with the uncheckedBounds
annotation (https://github.com/scala/scala/blob/v2.13.0-M2/src/compiler/scala/tools/nsc/typechecker/NamesDefaults.scala#L173-L189)
scala> :pa -raw
// Entering paste mode (ctrl-D to finish)
trait A
trait B
class Foo[A2, B2 <: A2] {
class Bar
object Bar {
implicit def fromString(a: String) = new Bar
}
def andThen(b : Bar) = b
}
object Test {
def lub = if (true) (null: Foo[A, A]) else (null: Foo[B, B])
(lub) andThen ("Bar")
}
// Exiting paste mode, now interpreting.
[[syntax trees at end of typer]] // <pastie>
def lub: Foo[_ >: A with B <: Object, _ >: A with B <: Object] = if (true)
(null: Foo[A,A])
else
(null: Foo[B,B]);
{
<synthetic> <stable> <artifact> val stabilizer$1: Foo[_1,_2] = Test.this.lub;
stabilizer$1.andThen(stabilizer$1.Bar.fromString("Bar"))
}
(lub) andThen ("Bar")
^
<pastie>:14: error: type arguments [_1,_2] do not conform to class Foo's type parameter bounds [A2,B2 <: A2]
There were compilation errors!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the pointer.
sjrd makes some good points at scala/bug#10619, which we should keep in mind while reasoning about this change. Also, does it interact ok with the various rewrites we do to method invocations? (by-name, prefix, defaults) I'm a little uneasy with this change, as we're basically adding a pretty ad-hoc form of wildcard capture (by giving the unstable prefix a name). Maybe that's ok, but we should think about this carefully in those terms. The Java type system and its rules around wildcard capture are pretty hard to follow. |
@adriaanm I don't think the points I make in scala/bug#10619 port over to this PR. There is one fundamental difference between this PR and the suggestions in the other issue: in this PR, we are only applying term rewritings that preserve evaluation order. In the other issues, the comments suggest to take a path from a type (where no evaluation order is even defined) and move it to a term position, thereby inventing some evaluation context for the term. I do not see any problem with this PR as far as semantics are concerned. |
test/files/pos/t4225.scala
Outdated
f.op1(23) | ||
f.op2(23)(true) | ||
f.op3[Int] | ||
f.op4(23) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could we introduce a run
test to show how by-name params interact with this lifting? I'd imagine that we'd need to lift a def or a val with a Function0
to preserve eval semantics.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If the qualifier is passed as a by-name arg to the implicit conversion, it doesn't seem suitably stable for a path (or equivalently, to be liftable to a val)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Agreed. That would have to be excluded.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's not actually the argument to the implicit conversion which must be strict, it's the argument of the extension method class. That ought to be prohibited as it stands, but it looks as though it's falling foul of scala/bug#10619. The following is a small reproduction for 2.12.4 and 2.13.x (without this PR) that's relevant here,
scala> class Foo { class Bar }
defined class Foo
scala> class Unsound[F <: Foo](f: => F) { def quux(a: f.Bar): f.Bar = a }
defined class Unsound
The two occurrences of f.Bar
in quux
should not be treated as having identical prefixes.
That's orthogonal to this PR though, and I can't add a neg test for the scenario you're worried about yet because it would be frustrated by scala/bug#10619.
I agree the intent preserves eval order, but, if I understood correctly, retronym identified one counter-example / needed exception. I'd like to see tests for all variations listed in my previous message (by-name, prefix, as well as defaults, just in case). For robustness, how about weird unstable things like a block that contains a definition? The other aspect of the change is that we are offering a limited form of wildcard capture with this rewrite. What if you have a long chain of method invocations that now results in a bunch of extra local vals that you don't want but can't disable? How about the recursion involved with rewriting chained calls? |
b3b2b4c
to
f3ccbbb
Compare
/rebuild |
There are some open comments: #5999 (review) and here #5999 (comment) |
f3ccbbb
to
0a1c62c
Compare
Also rebased. |
A stabilizing val is only inserted if necessary to make the term type check, so I don't think this is an issue. Prior to this PR the long chain you've described wouldn't compile without you manually inserting vals. Post this PR the long chain compiles, and there's nothing to prevent you from splitting it up as you would have been forced to previously. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM. Could you please:
- squash the changes into a single commit
- add a right-associative operator to the test for evaluation order, like:
scala> class Foo {
| class Bar
| object Bar {
| implicit def fromString(a: String) = new Bar
| }
| def andThen_:(b : Bar) = b
| }
scala> { println("bar"); "Bar" } andThen_: { println("foo"); new Foo }
bar
foo
res0: Foo#Bar = Foo$Bar@769bd849
scala> { println("foo"); new Foo }.andThen_:({ println("bar"); "Bar" })
foo
bar
res1: Foo#Bar = Foo$Bar@2e71240b
Oh, and also add a version with |
7c96009
to
084d1ac
Compare
@lrytz done. Note that your last request revealed a bug in byname arguments of right associative operators: scala/bug#10693. |
084d1ac
to
99c7778
Compare
Gah ... forgot the check file ... |
Could you update the commit message to add a short title line? https://github.com/scala/scala/blob/2.13.x/CONTRIBUTING.md#clean-commits-clean-history |
This fixes scala/bug#4225 and scala/bug#5946 by introducing a synthetic val for the unstable LHS of an implicit method application where the method type depends on the LHS. As applied to this example, taken from scala/bug#4225, ```scala class Foo { class Bar object Bar { implicit def mkBar: Bar = new Bar } } object f extends Foo implicit class Ops[F <: Foo](val f0: F) { def op(i: Int)(implicit b: f0.Bar): f0.Bar = b } f.op1(23) ``` this transformation produces, ```scala { val stabilizer$1 = Ops[f.type](f) stabilizer$1.op1(23) } ``` Because `stabilizer$1` is stable we retain the singleton type `F` and the implicit argument of type `f0.Bar` can be resolved against the implicit scope of `f.type`. A real world example of this is the macro API issue reported in scala/bug#5946 ... following this PR the usage below compiles as expected, ```scala class Ops(val g: scala.reflect.api.JavaUniverse) { def op[T: g.TypeTag] = () } implicit def Ops(g: scala.reflect.api.JavaUniverse): Ops = new Ops(g) scala.reflect.runtime.universe.op[Int] ``` In principle this transformation could also be applied for explicit applications, however there would no gain: to be able to write the explicit argument the user would already have had to explicitly introduce the necessary val.
99c7778
to
bcbe993
Compare
Done. Sorry about that ... I must have lost the title when I combined the messages. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you!
scala#5999 introduces a synthetic val for the unstable LHS of an implicit method application where the method type depends on the LHS. Unfortunately this only handles the case where an expression requires a _single_ stabilizing val: if the expression involves chained method applications, each requiring a stabilizing val, then only the last of these is emitted resulting in a failure during bytecode generation, object Test { class Foo { type Out } object Foo { implicit def foo: Foo { type Out = Bar } = ??? } class Bar { type Baz = Foo def foo(implicit foo: Baz): foo.Out = ??? } (new Bar).foo.foo } results in a NoSuchElementException, java.util.NoSuchElementException: value stabilizer$1 at scala.collection.mutable.AnyRefMap$ExceptionDefault.apply(AnyRefMap.scala:425) at scala.collection.mutable.AnyRefMap$ExceptionDefault.apply(AnyRefMap.scala:424) at scala.collection.mutable.AnyRefMap.apply(AnyRefMap.scala:180) at scala.tools.nsc.backend.jvm.BCodeSkelBuilder$PlainSkelBuilder$locals$.load(BCodeSkelBuilder.scala:392) ... Prior to this commit only a single stabilizer is recorded on the expression tree which is overwritten by later ones. This is fixed here by allowing multiple stabilizers to be recorded and emitted.
scala#5999 introduces a synthetic val for the unstable LHS of an implicit method application where the method type depends on the LHS. Unfortunately this only handles the case where an expression requires a _single_ stabilizing val: if the expression involves chained method applications, each requiring a stabilizing val, then only the last of these is emitted resulting in a failure during bytecode generation, object Test { class Foo { type Out } object Foo { implicit def foo: Foo { type Out = Bar } = ??? } class Bar { type Baz = Foo def foo(implicit foo: Baz): foo.Out = ??? } (new Bar).foo.foo } results in a NoSuchElementException, java.util.NoSuchElementException: value stabilizer$1 at scala.collection.mutable.AnyRefMap$ExceptionDefault.apply(AnyRefMap.scala:425) at scala.collection.mutable.AnyRefMap$ExceptionDefault.apply(AnyRefMap.scala:424) at scala.collection.mutable.AnyRefMap.apply(AnyRefMap.scala:180) at scala.tools.nsc.backend.jvm.BCodeSkelBuilder$PlainSkelBuilder$locals$.load(BCodeSkelBuilder.scala:392) ... Prior to this commit only a single stabilizer is recorded on the expression tree which is overwritten by later ones. This is fixed here by allowing multiple stabilizers to be recorded and emitted.
scala#5999 introduces a synthetic val for the unstable LHS of an implicit method application where the method type depends on the LHS. Unfortunately this only handles the case where an expression requires a _single_ stabilizing val: if the expression involves chained method applications, each requiring a stabilizing val, then only the last of these is emitted resulting in a failure during bytecode generation, object Test { class Foo { type Out } object Foo { implicit def foo: Foo { type Out = Bar } = ??? } class Bar { type Baz = Foo def foo(implicit foo: Baz): foo.Out = ??? } (new Bar).foo.foo } results in a NoSuchElementException, java.util.NoSuchElementException: value stabilizer$1 at scala.collection.mutable.AnyRefMap$ExceptionDefault.apply(AnyRefMap.scala:425) at scala.collection.mutable.AnyRefMap$ExceptionDefault.apply(AnyRefMap.scala:424) at scala.collection.mutable.AnyRefMap.apply(AnyRefMap.scala:180) at scala.tools.nsc.backend.jvm.BCodeSkelBuilder$PlainSkelBuilder$locals$.load(BCodeSkelBuilder.scala:392) ... Prior to this commit only a single stabilizer is recorded on the expression tree which is overwritten by later ones. This is fixed here by allowing multiple stabilizers to be recorded and emitted.
scala#5999 introduces a synthetic val for the unstable LHS of an implicit method application where the method type depends on the LHS. Unfortunately this only handles the case where an expression requires a _single_ stabilizing val: if the expression involves chained method applications, each requiring a stabilizing val, then only the last of these is emitted resulting in a failure during bytecode generation, object Test { class Foo { type Out } object Foo { implicit def foo: Foo { type Out = Bar } = ??? } class Bar { type Baz = Foo def foo(implicit foo: Baz): foo.Out = ??? } (new Bar).foo.foo } results in a NoSuchElementException, java.util.NoSuchElementException: value stabilizer$1 at scala.collection.mutable.AnyRefMap$ExceptionDefault.apply(AnyRefMap.scala:425) at scala.collection.mutable.AnyRefMap$ExceptionDefault.apply(AnyRefMap.scala:424) at scala.collection.mutable.AnyRefMap.apply(AnyRefMap.scala:180) at scala.tools.nsc.backend.jvm.BCodeSkelBuilder$PlainSkelBuilder$locals$.load(BCodeSkelBuilder.scala:392) ... Prior to this commit only a single stabilizer is recorded on the expression tree which is overwritten by later ones. This is fixed here by allowing multiple stabilizers to be recorded and emitted.
scala#5999 introduces a synthetic val for the unstable LHS of an implicit method application where the method type depends on the LHS. Unfortunately this only handles the case where an expression requires a _single_ stabilizing val: if the expression involves chained method applications, each requiring a stabilizing val, then only the last of these is emitted resulting in a failure during bytecode generation, object Test { class Bar { class Foo { type Out } object Foo { implicit def foo: Foo { type Out = Bar } = ??? } def foo(implicit foo: Foo): foo.Out = ??? } (new Bar).foo.foo } results in a NoSuchElementException, java.util.NoSuchElementException: value stabilizer$1 at scala.collection.mutable.AnyRefMap$ExceptionDefault.apply(AnyRefMap.scala:425) at scala.collection.mutable.AnyRefMap$ExceptionDefault.apply(AnyRefMap.scala:424) at scala.collection.mutable.AnyRefMap.apply(AnyRefMap.scala:180) at scala.tools.nsc.backend.jvm.BCodeSkelBuilder$PlainSkelBuilder$locals$.load(BCodeSkelBuilder.scala:392) ... Prior to this commit only a single stabilizer is recorded on the expression tree which is overwritten by later ones. This is fixed here by allowing multiple stabilizers to be recorded and emitted. In addition to that, previously spurious dependencies were picked up due to insufficient dealiasing. The following, object Test { class Bar { class Foo { type Out } object Foo { implicit def foo: Foo { type Out = Bar } = ??? } def foo(implicit foo: Foo): foo.Out = ??? } (new Bar).foo.foo } triggered the preceding error despite not having a true dependency which would require a stabilizer in the first place. The test for a dependent argument now dealiases to eliminate these spurious dependencies. Finally we avoid retyping stabilizing definitions when they are emitted, fixing scala/bug#10736.
This fixes scala/bug#4225 and scala/bug#5946 by introducing a synthetic val for the unstable LHS of an implicit method application where the method type depends on the LHS.
As applied to this example, taken from scala/bug#4225,
this transformation produces,
Because
fresh
is stable we retain the singleton typeF
and the implicit argument of typef0.Bar
can be resolved against the implicit scope off.type
.A real world example of this is the macro API issue reported in scala/bug#5946 ... following this PR the usage below compiles as expected,
In principle this transformation could also be applied for explicit applications, however there would no gain: to be able the write the explicit argument the user would already have had to explicitly introduce the necessary val.