Quasiquotes #2714

merged 11 commits into from Jul 10, 2013


None yet

3 participants

The Scala Programming Language member

Supersedes #2691

The Scala Programming Language member

/cc @adriaanm

densh and others added some commits Jul 8, 2013
@densh densh moves TreeBuilder into the parser
This is the first of the two patches to the parser necessary for
quasiquotes to function.

This one moves TreeBuilder from Global to the internals of the Parsers,
so that quasiquotes will be able to override it later to support some
corner cases arising from splicing (see the subsequent quasiquote commit
for more details).

Surprisingly enough, almost noone used TreeBuilder outside the parser,
and it was necessary to move just a couple of methods to TreeGen to
satisfy broken dependencies.
@densh densh extensibility hooks for parser
This is the second of the two patches to the parser necessary for
quasiquotes to function.

This one applies just a couple of minor changes to the way parser works,
so that quasiquotes will be able to extend it to support some
corner cases arising from splicing (see the subsequent quasiquote commit
for more details).
@densh densh adds the lookahead routine to the parser
Introduces a scoping operator used to temporarily look into the future.
Backs up scanner data before evaluating a block and restores it after.

Not used anywhere, only necessary for the upcoming quasiquote patch
in order to reliably detect and accordingly process holes in quasiquoted
Scala syntax.
@xeno-by xeno-by introduces extensibility hooks into the reifier
Quasiquoting macros are surprisingly similar to reifying macros, since
both take something and then produce Scala ASTs for that something.

Therefore the upcoming quasiquote patch reuses the vanilla reifier,
adjusting it in key points to enable splicing and extraction to
support string interpolation syntax.

In this commit we prepare the reifier for being reused later on,
adding a modest amound of extensibility hooks.
@densh densh moves template creation logic from nsc to reflect
This routine is going to be necessary in scala-reflect.jar to support
ClassDef construction/deconstruction in the upcoming quasiquote patch.
@densh densh tests for quasiquotes
Introduces an extensive ScalaCheck-based test suite for recently
implemented quasiquotes. Provides tools for syntactic tree comparison
and verifying compilation error messages.
@xeno-by xeno-by introduces unapply macros for internal use
Adds a macro hook into the unapply part of `doTypedApply`, provides
`macroExpandUnapply` in the macro engine and applies a couple of minor
refactorings along the way (renames the ugly `macroExpand1` into
oblivion, changes the ctor of `Fingerprint` to be private and upgrades
`MacroRole` from a string to a value class).

Unapply macros haven't been approved for inclusion in 2.11.0, however
they are necessary for pattern-matching quasiquotes. Therefore I'm only
allowing them to expand for QuasiquoteClass_api_unapply.
@densh densh implements quasiquotes
- Additions to the reflection API:

  - The Quasiquotes implicit class that defines `q`, `tq`, `pq` and `cq`
    interpolators which now become a part of the `scala.reflect.api.

  - Implementations of the interpolators are macro-based
    and are hardwired through `FastTrack`.

  - The `Liftable` class and the `StandardLiftables` slice of the cake
    that provide a type class and a bunch of its instances that allow
    to easily splice user-defined types into quasiquotes.

  - Additional methods in `BuildUtils` that are used by the quasiquote
    macro to generate trees, notably:

    - `SyntacticClassDef`. An extractor/constructor that allows to
      construct and deconstruct classes using arguments that mirror
      syntactic form of ClassDefs (e.g. constructor outside of the

    - `TupleN`, `TupleTypeN`. Extractor/constructor for easy
      construction of ast that represents a tuple term or type
      with given amount of elements.

- Actual implementation of quasiquotes in the `scala.tools.reflect.
  quasiquotes` package which is organized into a cake called
  `Quasiquotes` with slices introducing core abstractions necessary to
  splice into Scala syntax, routines for interfacing with the parser,
  and customized reifiers for tree construction and deconstruction.
The Scala Programming Language member

Copy/paste removed, quasiquote engine refactored to be much more clear, other comments from the previous pull request addressed. LGTM

The Scala Programming Language member

Ok, I'll review this afternoon. M4 is scheduled for tomorrow, so in principle this can go in.

The Scala Programming Language member

for backwards compatibility, could we leave a forwarder def treeBuilder: TreeBuilder = parser.treeBuilder?

The Scala Programming Language member


The Scala Programming Language member

this could really use a comment

The Scala Programming Language member

how about a comment with a snippet that shows how to use this trait?

The Scala Programming Language member

classes in an api package should really have some (minimal) documentation

The Scala Programming Language member

Change return type to Some[Long] to help the pattern matcher analyze matches better.

The Scala Programming Language member


The Scala Programming Language member



could we get some BNF-style syntax for the modified fragment of the language?

The Scala Programming Language member

Excellent, thanks for splitting it up nicely!

I think this can go in M4 (I plan to tag my Wednesday morning). I listed some suggestions below that would be nice, time permitting -- I'd definitely like to see them addressed at some point, if not by M4.

My (ordered) wishlist:

  • if possible, leave a forwarder to treeBuilder in the parser for potential plugin users (or other compiler clients).
  • add a little scaladoc to the undocumented classes/traits that are introduced in reflect.api
  • a comment in the parser on why the def case now comes before the expr case when parsing a template
  • BNF-style syntax description of the modified fragment of Scala implemented by QuasiquoteParser
densh added some commits Jul 10, 2013
@densh densh backward compatibility for TreeBuilder
Reintroduces treeBuilder at its old location (Global.treeBuilder) by
routing it from the new location (Parsers.treeBuilder) for the sake of
being friendly to tools that might depend on it.
@densh densh precise return type for FlagsAsBits.unapply
This will help the pattern matcher to emit better code for this kind
of extractor that does nothing but wrap the extractee.
The Scala Programming Language member

@adriaanm As I was testing out quasiquotes, making sure that things work well for M4, I discovered that relaxed macro impls (ones that allow their arguments and return types to be trees) don't work with subtypes of c.Tree, just with c.Tree itself. Since quasiquotes don't upcast their expansion to c.Tree, this causes problems.

I thought this might be a very common stumbling point, so I took liberty to fix the bug and append the patch to this pull request, given that it's really small. Please let me know if you want me to resubmit the fix separarely.

The Scala Programming Language member

Should be fine. I'll have a look in the next couple of hours

@xeno-by xeno-by macro impls can now return subtypes of c.Tree
1022931 allowed macro impls to take and return values of type c.Tree
in addition to the usual c.Expr. However it didn't take into account that
it is often useful to return subtypes of trees (e.g. with quasiquotes
that expand into expressions typed as precisely as possible). This patch
fixes that oversight.
The Scala Programming Language member

The failures were caused by the fix to SI-7507 not been applied to locker. When fixing the macro engine, I also did some code cleanup and removed a workaround for SI-7507. The patch compiled fine on my machine when I was working with quick, however it caused errors when jenkins tried to build locker. I removed the cleanup commit and pushed -f.

The Scala Programming Language member

LGTM, great work, guys! 🍰 for everyone.

I will be back to insist on those missing docs, though.

@adriaanm adriaanm merged commit 9def2ad into scala:master Jul 10, 2013

1 check passed

Details default pr-checkin-per-commit Took 46 min.
The Scala Programming Language member

The check-init build discovered some likely initialization bugs: https://scala-webapps.epfl.ch/jenkins/view/All/job/scala-nightly-checkinit/1761/console:

 Uninitialized field: Reifiers.scala: 19
    at scala.tools.reflect.quasiquotes.Reifiers$Reifier.reifee(Reifiers.scala:19)
    at scala.tools.reflect.quasiquotes.Reifiers$Reifier.reifee(Reifiers.scala:16)
    at scala.reflect.reify.States$State.<init>(States.scala:43)
    at scala.reflect.reify.States$class.$init$(States.scala:22)
    at scala.reflect.reify.Reifier.<init>(Reifier.scala:15)
    at scala.tools.reflect.quasiquotes.Reifiers$Reifier.<init>(Reifiers.scala:18)
    at scala.tools.reflect.quasiquotes.Reifiers$ApplyReifier.<init>(Reifiers.scala:166)
    at scala.tools.reflect.quasiquotes.Quasiquotes$$anonfun$2.apply(Quasiquotes.scala:24)
    at scala.tools.reflect.quasiquotes.Quasiquotes$$anonfun$2.apply(Quasiquotes.scala:24)
    at scala.tools.reflect.quasiquotes.Quasiquotes.expandQuasiquote(Quasiquotes.scala:47)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment