Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Loading…

SI-8331 make sure type select & applied type doesn't match terms #3594

Merged
merged 2 commits into from

4 participants

Denys Shabalin Jason Zaugg Eugene Burmako Adriaan Moors
Denys Shabalin
Owner

Due to tree re-use it used to be the fact that type quasiquotes could
match term trees. This commit makes sure selections and applied type and
type applied are all non-overlapping between q and tq.

review @retronym

Denys Shabalin densh added this to the 2.11.0-RC2 milestone
Jason Zaugg retronym was assigned by densh
...flect/scala/reflect/internal/ReificationSupport.scala
@@ -219,7 +231,10 @@ trait ReificationSupport { self: SymbolTable =>
case UnApply(treeInfo.Unapplied(Select(fun, nme.unapply)), pats) =>
Some((fun, pats :: Nil))
case treeInfo.Applied(fun, targs, argss) =>
- Some((SyntacticTypeApplied(fun, targs), argss))
+ if (fun.isTerm)
+ Some((SyntacticTypeApplied(fun, targs), argss))
+ else
+ Some((SyntacticAppliedType(fun, targs), argss))
Jason Zaugg Owner

You could tighten up the if else here to avoid some nanoscale duplication

Denys Shabalin Owner
densh added a note

Good idea.

Denys Shabalin Owner
densh added a note

Fixed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Jason Zaugg
Owner

Could you elaborate on how tree reuse leads to the bug, perhaps with an example?

Denys Shabalin
Owner

Sure. Lets say we have a user that has no idea about internal organization of trees but have read a quasiquote guide and decides to write a tree transformer that wraps all term selections into a magical call:

class MagicTransformer extends Transformer {
  override transform(tree: Tree) = tree match {
    case q"$qual.$name" => 
      val transformedQual = transform(qual)
      q"magic($transformedQual, ${name.toString})"
    case _ => super.transform(tree)
  }
}

This code might look correct at first but if we look at the desugaring of the pattern we'll see:

case Select(qual, name) =>

Which will match both term and type selections. Definitely not something one would expect when they probably wanted to match just terms (due to usage of q"" rather than tq"")

Similarly with type application / applied type. One would probably expect q"" to only match type applications and tq"" to only match applied types.

Denys Shabalin densh added tested and removed tested labels
Eugene Burmako
Owner

Needs to be rebased.

Eugene Burmako
Owner

Can something be done here for idents as well?

...flect/scala/reflect/internal/ReificationSupport.scala
((10 lines not shown))
case TypeApply(fun, targs) => Some((fun, targs))
+ case _ if tree.isTerm => Some((tree, Nil))
+ case _ => None
+ }
+ }
+
+ object SyntacticAppliedType extends SyntacticTypeAppliedExtractor {
Eugene Burmako Owner
xeno-by added a note

I think this would use a comment that explains its purpose.

Denys Shabalin Owner
densh added a note

Fixed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
...flect/scala/reflect/internal/ReificationSupport.scala
@@ -219,7 +231,10 @@ trait ReificationSupport { self: SymbolTable =>
case UnApply(treeInfo.Unapplied(Select(fun, nme.unapply)), pats) =>
Some((fun, pats :: Nil))
case treeInfo.Applied(fun, targs, argss) =>
- Some((SyntacticTypeApplied(fun, targs), argss))
+ val inbrackets =
Eugene Burmako Owner
xeno-by added a note

inbrackets? What does this name mean?

Denys Shabalin Owner
densh added a note

fun[targs], targs are in brackets.

Eugene Burmako Owner
xeno-by added a note

But isn't inbrackets going to contain the entire fun[targs] thing?

Denys Shabalin Owner
densh added a note

That's just a temp variable name that's better than tree.

Eugene Burmako Owner
xeno-by added a note

Don't think it's better given that it's misleading. Could you rename?

Denys Shabalin Owner
densh added a note

Into tree?

Denys Shabalin Owner
densh added a note

Fixed.

Eugene Burmako Owner
xeno-by added a note

Very funny. Not.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Denys Shabalin
Owner

Can something be done here for idents as well?

Idents are handled by Liftable. You can't fix it without redesigning the api, possibly by splitting into multiple type classes for terms, types and patterns.

Denys Shabalin densh added tested and removed tested labels
...flect/scala/reflect/internal/ReificationSupport.scala
@@ -223,7 +241,10 @@ trait ReificationSupport { self: SymbolTable =>
case UnApply(treeInfo.Unapplied(Select(fun, nme.unapply)), pats) =>
Some((fun, pats :: Nil))
case treeInfo.Applied(fun, targs, argss) =>
- Some((SyntacticTypeApplied(fun, targs), argss))
+ val `fun[targs]` =
Eugene Burmako Owner
xeno-by added a note

result? callee?

Denys Shabalin Owner
densh added a note

Done.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Denys Shabalin densh removed the tested label
Eugene Burmako
Owner

LGTM

Eugene Burmako
Owner

@adriaanm This pull request isn't binary compatible with RC1. Can we merge nonetheless, because: a) it is definitely useful, b) the incompatibility consists in scala.reflect.api#internal and affects only quasiquotes?

Denys Shabalin
Owner

ping @retronym.

Eugene Burmako
Owner

We need to rebuild this pull request in order to expose and fix the binary incompat that it introduces.

Jason Zaugg
Owner

LGTM

densh added some commits
Denys Shabalin densh SI-8331 make sure type select & applied type doesn't match terms
Due to tree re-use it used to be the fact that type quasiquotes could
match term trees. This commit makes sure selections and applied type and
type applied are all non-overlapping between q and tq.
67d175f
Denys Shabalin densh Address pull request feedback
1. Tighten up the if else to avoid duplication
2. Add doc comments
1b5a34b
Denys Shabalin
Owner

Rebased to uncover bincompat issues. Added proposed changes to whitelists.

Denys Shabalin densh added tested and removed tested labels
Adriaan Moors
Owner

LGTM, as discussed during scala meeting

Adriaan Moors adriaanm merged commit 51a413e into from
Adriaan Moors adriaanm modified the milestone: 2.11.0-RC3, 2.11.0-RC2
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Commits on Mar 9, 2014
  1. Denys Shabalin

    SI-8331 make sure type select & applied type doesn't match terms

    densh authored
    Due to tree re-use it used to be the fact that type quasiquotes could
    match term trees. This commit makes sure selections and applied type and
    type applied are all non-overlapping between q and tq.
  2. Denys Shabalin

    Address pull request feedback

    densh authored
    1. Tighten up the if else to avoid duplication
    2. Add doc comments
This page is out of date. Refresh to see the latest.
25 bincompat-backward.whitelist.conf
View
@@ -4,8 +4,8 @@ filter {
# "scala.concurrent.impl"
# "scala.reflect.runtime"
]
- // see SI-8372
problems=[
+ // see SI-8372
{
matchName="scala.collection.mutable.ArrayOps#ofChar.unzip"
problemName=IncompatibleMethTypeProblem
@@ -101,6 +101,27 @@ filter {
{
matchName="scala.collection.mutable.ArrayOps#ofDouble.unzip3"
problemName=IncompatibleMethTypeProblem
+ },
+ // see SI-8331
+ {
+ matchName="scala.reflect.api.Internals#ReificationSupportApi#SyntacticTypeAppliedExtractor.unapply"
+ problemName=IncompatibleResultTypeProblem
+ },
+ {
+ matchName="scala.reflect.api.Internals#ReificationSupportApi#SyntacticTypeAppliedExtractor.unapply"
+ problemName=MissingMethodProblem
+ },
+ {
+ matchName="scala.reflect.api.Internals#ReificationSupportApi.SyntacticSelectType"
+ problemName=MissingMethodProblem
+ },
+ {
+ matchName="scala.reflect.api.Internals#ReificationSupportApi.SyntacticAppliedType"
+ problemName=MissingMethodProblem
+ },
+ {
+ matchName="scala.reflect.api.Internals#ReificationSupportApi.SyntacticSelectTerm"
+ problemName=MissingMethodProblem
}
]
-}
+}
31 bincompat-forward.whitelist.conf
View
@@ -4,8 +4,8 @@ filter {
# "scala.concurrent.impl"
# "scala.reflect.runtime"
]
- // see SI-8372
problems=[
+ // see SI-8372
{
matchName="scala.collection.mutable.ArrayOps#ofChar.unzip"
problemName=IncompatibleMethTypeProblem
@@ -101,6 +101,35 @@ filter {
{
matchName="scala.collection.mutable.ArrayOps#ofDouble.unzip3"
problemName=IncompatibleMethTypeProblem
+ },
+ // see SI-8331
+ {
+ matchName="scala.reflect.api.Internals#ReificationSupportApi.SyntacticSelectType"
+ problemName=MissingMethodProblem
+ },
+ {
+ matchName="scala.reflect.api.Internals#ReificationSupportApi.SyntacticAppliedType"
+ problemName=MissingMethodProblem
+ },
+ {
+ matchName="scala.reflect.api.Internals#ReificationSupportApi.SyntacticSelectTerm"
+ problemName=MissingMethodProblem
+ },
+ {
+ matchName="scala.reflect.api.Internals$ReificationSupportApi$SyntacticSelectTermExtractor"
+ problemName=MissingClassProblem
+ },
+ {
+ matchName="scala.reflect.api.Internals#ReificationSupportApi#SyntacticTypeAppliedExtractor.unapply"
+ problemName=IncompatibleResultTypeProblem
+ },
+ {
+ matchName="scala.reflect.api.Internals#ReificationSupportApi#SyntacticTypeAppliedExtractor.unapply"
+ problemName=MissingMethodProblem
+ },
+ {
+ matchName="scala.reflect.api.Internals$ReificationSupportApi$SyntacticSelectTypeExtractor"
+ problemName=MissingClassProblem
}
]
}
5 src/compiler/scala/tools/reflect/quasiquotes/Reifiers.scala
View
@@ -185,6 +185,8 @@ trait Reifiers { self: Quasiquotes =>
reifyBuildCall(nme.SyntacticApplied, fun, argss)
case SyntacticTypeApplied(fun, targs) if targs.nonEmpty =>
reifyBuildCall(nme.SyntacticTypeApplied, fun, targs)
+ case SyntacticAppliedType(tpt, targs) if targs.nonEmpty =>
+ reifyBuildCall(nme.SyntacticAppliedType, tpt, targs)
case SyntacticFunction(args, body) =>
reifyBuildCall(nme.SyntacticFunction, args, body)
case SyntacticIdent(name, isBackquoted) =>
@@ -215,6 +217,9 @@ trait Reifiers { self: Quasiquotes =>
// correctness of the trees produced by quasiquotes
case Select(id @ Ident(nme.scala_), name) if id.symbol == ScalaPackage =>
reifyBuildCall(nme.ScalaDot, name)
+ case Select(qual, name) =>
+ val ctor = if (name.isTypeName) nme.SyntacticSelectType else nme.SyntacticSelectTerm
+ reifyBuildCall(ctor, qual, name)
case _ =>
super.reifyTreeSyntactically(tree)
}
15 src/reflect/scala/reflect/api/Internals.scala
View
@@ -600,10 +600,11 @@ trait Internals { self: Universe =>
}
val SyntacticTypeApplied: SyntacticTypeAppliedExtractor
+ val SyntacticAppliedType: SyntacticTypeAppliedExtractor
trait SyntacticTypeAppliedExtractor {
def apply(tree: Tree, targs: List[Tree]): Tree
- def unapply(tree: Tree): Some[(Tree, List[Tree])]
+ def unapply(tree: Tree): Option[(Tree, List[Tree])]
}
val SyntacticApplied: SyntacticAppliedExtractor
@@ -784,6 +785,18 @@ trait Internals { self: Universe =>
def apply(expr: Tree, selectors: List[Tree]): Import
def unapply(imp: Import): Some[(Tree, List[Tree])]
}
+
+ val SyntacticSelectType: SyntacticSelectTypeExtractor
+ trait SyntacticSelectTypeExtractor {
+ def apply(qual: Tree, name: TypeName): Select
+ def unapply(tree: Tree): Option[(Tree, TypeName)]
+ }
+
+ val SyntacticSelectTerm: SyntacticSelectTermExtractor
+ trait SyntacticSelectTermExtractor {
+ def apply(qual: Tree, name: TermName): Select
+ def unapply(tree: Tree): Option[(Tree, TermName)]
+ }
}
@deprecated("Use `internal.reificationSupport` instead", "2.11.0")
53 src/reflect/scala/reflect/internal/ReificationSupport.scala
View
@@ -101,7 +101,7 @@ trait ReificationSupport { self: SymbolTable =>
}
def mkAnnotation(tree: Tree): Tree = tree match {
- case SyntacticNew(Nil, SyntacticApplied(SyntacticTypeApplied(_, _), _) :: Nil, noSelfType, Nil) =>
+ case SyntacticNew(Nil, SyntacticApplied(SyntacticAppliedType(_, _), _) :: Nil, noSelfType, Nil) =>
tree
case _ =>
throw new IllegalArgumentException(s"Tree ${showRaw(tree)} isn't a correct representation of annotation." +
@@ -201,17 +201,35 @@ trait ReificationSupport { self: SymbolTable =>
def unapply(flags: Long): Some[Long] = Some(flags)
}
+ /** Construct/deconstruct type application term trees.
+ * Treats other term trees as zero-argument type applications.
+ */
object SyntacticTypeApplied extends SyntacticTypeAppliedExtractor {
def apply(tree: Tree, targs: List[Tree]): Tree =
if (targs.isEmpty) tree
else if (tree.isTerm) TypeApply(tree, targs)
- else if (tree.isType) AppliedTypeTree(tree, targs)
- else throw new IllegalArgumentException(s"can't apply types to $tree")
+ else throw new IllegalArgumentException(s"can't apply type arguments to $tree")
- def unapply(tree: Tree): Some[(Tree, List[Tree])] = tree match {
+ def unapply(tree: Tree): Option[(Tree, List[Tree])] = tree match {
case TypeApply(fun, targs) => Some((fun, targs))
+ case _ if tree.isTerm => Some((tree, Nil))
+ case _ => None
+ }
+ }
+
+ /** Construct/deconstruct applied type trees.
+ * Treats other types as zero-arity applied types.
+ */
+ object SyntacticAppliedType extends SyntacticTypeAppliedExtractor {
+ def apply(tree: Tree, targs: List[Tree]): Tree =
+ if (targs.isEmpty) tree
+ else if (tree.isType) AppliedTypeTree(tree, targs)
+ else throw new IllegalArgumentException(s"can't create applied type from non-type $tree")
+
+ def unapply(tree: Tree): Option[(Tree, List[Tree])] = tree match {
case AppliedTypeTree(tpe, targs) => Some((tpe, targs))
- case _ => Some((tree, Nil))
+ case _ if tree.isType => Some((tree, Nil))
+ case _ => None
}
}
@@ -223,7 +241,10 @@ trait ReificationSupport { self: SymbolTable =>
case UnApply(treeInfo.Unapplied(Select(fun, nme.unapply)), pats) =>
Some((fun, pats :: Nil))
case treeInfo.Applied(fun, targs, argss) =>
- Some((SyntacticTypeApplied(fun, targs), argss))
+ val callee =
+ if (fun.isTerm) SyntacticTypeApplied(fun, targs)
+ else SyntacticAppliedType(fun, targs)
+ Some((callee, argss))
}
}
@@ -489,8 +510,8 @@ trait ReificationSupport { self: SymbolTable =>
gen.mkNew(parents, mkSelfType(selfType), earlyDefs ::: body, NoPosition, NoPosition)
def unapply(tree: Tree): Option[(List[Tree], List[Tree], ValDef, List[Tree])] = tree match {
- case SyntacticApplied(Select(New(SyntacticTypeApplied(ident, targs)), nme.CONSTRUCTOR), argss) =>
- Some((Nil, SyntacticApplied(SyntacticTypeApplied(ident, targs), argss) :: Nil, noSelfType, Nil))
+ case SyntacticApplied(Select(New(SyntacticAppliedType(ident, targs)), nme.CONSTRUCTOR), argss) =>
+ Some((Nil, SyntacticApplied(SyntacticAppliedType(ident, targs), argss) :: Nil, noSelfType, Nil))
case SyntacticBlock(SyntacticClassDef(_, tpnme.ANON_CLASS_NAME, Nil, _, ListOfNil, earlyDefs, parents, selfType, body) ::
Apply(Select(New(Ident(tpnme.ANON_CLASS_NAME)), nme.CONSTRUCTOR), Nil) :: Nil) =>
Some((earlyDefs, parents, selfType, body))
@@ -987,6 +1008,22 @@ trait ReificationSupport { self: SymbolTable =>
Some((imp.expr, selectors))
}
}
+
+ object SyntacticSelectType extends SyntacticSelectTypeExtractor {
+ def apply(qual: Tree, name: TypeName): Select = Select(qual, name)
+ def unapply(tree: Tree): Option[(Tree, TypeName)] = tree match {
+ case Select(qual, name: TypeName) => Some((qual, name))
+ case _ => None
+ }
+ }
+
+ object SyntacticSelectTerm extends SyntacticSelectTermExtractor {
+ def apply(qual: Tree, name: TermName): Select = Select(qual, name)
+ def unapply(tree: Tree): Option[(Tree, TermName)] = tree match {
+ case Select(qual, name: TermName) => Some((qual, name))
+ case _ => None
+ }
+ }
}
val build = new ReificationSupportImpl
3  src/reflect/scala/reflect/internal/StdNames.scala
View
@@ -613,6 +613,7 @@ trait StdNames {
val SingleType: NameType = "SingleType"
val SuperType: NameType = "SuperType"
val SyntacticApplied: NameType = "SyntacticApplied"
+ val SyntacticAppliedType: NameType = "SyntacticAppliedType"
val SyntacticAssign: NameType = "SyntacticAssign"
val SyntacticBlock: NameType = "SyntacticBlock"
val SyntacticClassDef: NameType = "SyntacticClassDef"
@@ -630,6 +631,8 @@ trait StdNames {
val SyntacticObjectDef: NameType = "SyntacticObjectDef"
val SyntacticPackageObjectDef: NameType = "SyntacticPackageObjectDef"
val SyntacticPatDef: NameType = "SyntacticPatDef"
+ val SyntacticSelectType: NameType = "SyntacticSelectType"
+ val SyntacticSelectTerm: NameType = "SyntacticSelectTerm"
val SyntacticTraitDef: NameType = "SyntacticTraitDef"
val SyntacticTry: NameType = "SyntacticTry"
val SyntacticTuple: NameType = "SyntacticTuple"
12 test/files/scalacheck/quasiquotes/TermDeconstructionProps.scala
View
@@ -199,4 +199,16 @@ object TermDeconstructionProps extends QuasiquoteProperties("term deconstruction
q"..$tpt; ()"
}
}
+
+ property("term select doesn't match type select") = test {
+ assertThrows[MatchError] {
+ val q"$qual.$name" = tq"foo.bar"
+ }
+ }
+
+ property("type application doesn't match applied type") = test {
+ assertThrows[MatchError] {
+ val q"$f[..$targs]" = tq"foo[bar]"
+ }
+ }
}
12 test/files/scalacheck/quasiquotes/TypeDeconstructionProps.scala
View
@@ -63,4 +63,16 @@ object TypeDeconstructionProps extends QuasiquoteProperties("type deconstruction
// matches because type tree isn't syntactic without original
val tq"" = tq"${typeOf[Int]}"
}
+
+ property("type select doesn't match term select") = test {
+ assertThrows[MatchError] {
+ val tq"$qual.$name" = q"foo.bar"
+ }
+ }
+
+ property("applied type doesn't match type appliction") = test {
+ assertThrows[MatchError] {
+ val tq"$tpt[..$tpts]" = q"foo[bar]"
+ }
+ }
}
Something went wrong with that request. Please try again.