Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Already on GitHub? Sign in to your account

Add tree-based code generation #3321

Merged
merged 8 commits into from Jan 15, 2014

Conversation

Projects
None yet
5 participants
Contributor

VladimirNik commented Dec 31, 2013

New printer (ParsedTreePrinter) is intended for unattributed trees and generates the code based on the passed AST.
Result code can be later compiled by scalac retaining the same meaning.
ParsedTreePrinter is based on Sprinter project.

VladimirNik added some commits Dec 31, 2013

Variance annotations printing
def printTypeParams is modified. Tests are updated.
val showOuterTests is removed
val showOuterTests wasn't used in Printers

@ghost ghost assigned retronym Jan 1, 2014

Member

xeno-by commented Jan 1, 2014

Yay! Happy New Year!

Member

xeno-by commented Jan 1, 2014

Great job! It's very cool to see this kind of effort, and I'm looking forward to having toCode in trunk! Review @retronym /cc @densh

Owner

retronym commented Jan 1, 2014

Similar functionality exists over in scala-refactoring:

https://github.com/scala-ide/scala-refactoring/blob/master/org.scala-refactoring.library/src/main/scala/scala/tools/refactoring/sourcegen/PrettyPrinter.scala

Could you compare please make sure that your test suite covers at least the cases in PrettyPrinterTest ?

/cc @misto

Contributor

VladimirNik commented Jan 3, 2014

@retronym Yes, as far as I know tree printing in scala-refactoring is targeted on typed trees produced by presentation compiler. Our target in Sprinter was to generate semantically equivalent code using mainly tree structure (we used only dependency on scala.reflect for our main printer), that's why there's a lot of desugaring in result code and we plan to improve it in the future.

I added to my test suite new tests with some code snippets from scala-refactoring tests. Also to test new ParsedTreePrinter I used scalaz (scalaz sources were regenerated using printPlugin, recompiled, and then all project's tests were passed successfully).

VladimirNik added some commits Jan 5, 2014

toCode is added to Printers
1. Printers API is updated (def toCode)
2. ParsedTreePrinter is added to internal.Printers. ParsedTreePrinter generates the code for passed unattributed trees.
Generated code can be later compiled by scalac retaining the same meaning.
Contributor

VladimirNik commented Jan 5, 2014

Version with fixes from comments is added. Commits are updated.

@densh densh and 2 others commented on an outdated diff Jan 6, 2014

src/reflect/scala/reflect/internal/Printers.scala
case anns => anns
}
- annots foreach (annot => print("@"+annot+" "))
+ annots foreach (annot => print("@" + annot + " "))
@densh

densh Jan 6, 2014

Contributor

s"@$annot " ?

@retronym

retronym Jan 6, 2014

Owner

More to the point, does annotation.toString always produce re-typecheckable code? What about back-ticks for special characters? How are the arguments to the annotation printed?

@VladimirNik

VladimirNik Jan 7, 2014

Contributor

@densh Code fixed
@retronym It's a version of printAnnotations method for TreePrinter. ParsedTreePrinter overrides this method and process back-ticks for special characters. In the case of TreePrinter arguments to the annotation are printed as a part of tree because toString results in invocation of TreePrinter's printTree(annot).

@densh densh and 1 other commented on an outdated diff Jan 6, 2014

src/reflect/scala/reflect/internal/Printers.scala
@@ -459,11 +525,496 @@ trait Printers extends api.Printers { self: SymbolTable =>
out.print(if (arg == null) "null" else arg.toString)
}
}
+
+ //printer for trees after parser and before typer phases
@densh

densh Jan 6, 2014

Contributor

Maybe one more space after // and before beginning of the comment?

@VladimirNik

VladimirNik Jan 7, 2014

Contributor

Fixed

You wanted to say "TODO: ..."? Some clarifications of the future plans would be of help here.

Owner

VladimirNik replied Jan 7, 2014

Fixed

@densh densh and 1 other commented on an outdated diff Jan 6, 2014

src/reflect/scala/reflect/internal/Printers.scala
+ if (name.startsWith(nme.WHILE_PREFIX)) {
+ val If(cond, thenp, elsep) = rhs
+ print("while (", cond, ") ")
+ val Block(list, wh) = thenp
+ printColumn(list, "", ";", "")
+ } else if (name.startsWith(nme.DO_WHILE_PREFIX)) {
+ val Block(bodyList, ifCond @ If(cond, thenp, elsep)) = rhs
+ print("do ")
+ printColumn(bodyList, "", ";", "")
+ print(" while (", cond, ") ")
+ } else {
+ print(printedName(name)); printLabelParams(params);
+ printBlock(rhs)
+ }
+
+ case imp @ Import(expr, selectors) =>
@densh

densh Jan 6, 2014

Contributor

selectors not used? use _ instead?

@VladimirNik

VladimirNik Jan 7, 2014

Contributor

Fixed

@densh densh and 1 other commented on an outdated diff Jan 6, 2014

src/reflect/scala/reflect/internal/Printers.scala
@@ -104,9 +102,14 @@ trait Printers extends api.Printers { self: SymbolTable =>
def printRow(ts: List[Tree], sep: String) { printRow(ts, "", sep, "") }
def printTypeParams(ts: List[TypeDef]) {
@densh

densh Jan 6, 2014

Contributor

Procedure syntax is planned to be deprecated in the future, use def f: Unit = ... instead.

@VladimirNik

VladimirNik Jan 7, 2014

Contributor

Fixed

Contributor

densh commented Jan 6, 2014

I see a lot of promise in this work. There are a few low hanging fruits that could be improved by using Syntactic extractors but it would be extremely easy to add them as follow-ups later on.

@densh densh and 1 other commented on an outdated diff Jan 6, 2014

src/reflect/scala/reflect/internal/Printers.scala
+ case Apply(Block(l1 @ List(sVD: ValDef), a1 @ Apply(Select(_, methodName), l2 @ List(Ident(iVDName)))), l3)
+ if sVD.mods.isSynthetic && treeInfo.isLeftAssoc(methodName) && sVD.name == iVDName =>
+ val printBlock = Block(l1, Apply(a1, l3))
+ print(printBlock)
+ case Apply(tree1, _) if (needsParentheses(tree1)(insideAnnotated = false)) =>
+ parenthesize()(print(fun)); printRow(vargs, "(", ", ", ")")
+ case _ => super.printTree(tree)
+ }
+
+ case st @ Super(This(qual), mix) =>
+ printSuper(st, printedName(qual))
+
+ case th @ This(qual) =>
+ printThis(th, printedName(qual))
+
+ case Select(qual @ New(tpe), name) =>
@densh

densh Jan 6, 2014

Contributor

tpe not used, maybe qual: New subpattern instead?

@VladimirNik

VladimirNik Jan 7, 2014

Contributor

Fixed

@retronym retronym and 1 other commented on an outdated diff Jan 6, 2014

src/reflect/scala/reflect/internal/Printers.scala
+
+ protected def printSuper(tree: Super, resultName: => String) {
+ val Super(This(qual), mix) = tree
+ if (qual.nonEmpty || tree.symbol != NoSymbol) print(resultName + ".")
+ print("super")
+ if (mix.nonEmpty) print(s"[$mix]")
+ }
+
+ protected def printThis(tree: This, resultName: => String) {
+ val This(qual) = tree
+ if (qual.nonEmpty) print(resultName + ".")
+ print("this")
+ }
+
+ protected def printAnnotated(tree: Annotated)(printArg: => Unit) {
+ val Annotated(Apply(Select(New(tpt), nme.CONSTRUCTOR), args), atree) = tree
@retronym

retronym Jan 6, 2014

Owner

Does this handle annotations with multiple parameter lists?

@VladimirNik

VladimirNik Jan 7, 2014

Contributor

I resolved this issue, new version handles Annotated trees that contain annotations with multiple parameter lists.

VladimirNik added some commits Jan 6, 2014

Annotated trees processing is modified
1. Problem with multiple parameter lists in annotations is resolved
2. Tests for Annotated trees are added
Contributor

VladimirNik commented Jan 7, 2014

@xeno-by @densh @retronym Thanks for the comments! Fixed version is committed.

Member

xeno-by commented Jan 12, 2014

@VladimirNik You know, I've been thinking now, and how about we turn Tree.toCode into Universe.showCode in order to be consistent with show and showRaw (https://github.com/xeno-by/scala/blob/master/src/reflect/scala/reflect/api/Printers.scala#L196)?

Contributor

VladimirNik commented Jan 12, 2014

@xeno-by I agree, the only thing - before we had toCode in Printers (and could access it from Universe, not Tree). In my last commit toCode(tree: Tree) is changed to showCode(tree: Tree)

Owner

adriaanm commented Jan 14, 2014

awesome! could one of the reviewers sign off with an LGTM? (remember, it has to be the first word of your comment in order to count)

Member

xeno-by commented Jan 15, 2014

LGTM

Member

xeno-by commented Jan 15, 2014

Shall we? :)

retronym added a commit that referenced this pull request Jan 15, 2014

Merge pull request #3321 from VladimirNik/sprinter
Add tree-based code generation

@retronym retronym merged commit 07e823a into scala:master Jan 15, 2014

1 check passed

default pr-scala Took 53 min.
Details
Contributor

densh commented Jan 15, 2014

🍰 to the @VladimirNik, keep up the good stuff.

Owner

adriaanm commented Jan 16, 2014

Cool stuff indeed, but it breaks on windows unfortunately: https://scala-webapps.epfl.ch/jenkins/job/scala-nightly-windows/1945/console

Owner

adriaanm commented Jan 16, 2014

Since we unfortunately don't archive test output on nightlies, I quickly went ahead and capture the relevant log file: https://gist.github.com/adriaanm/8449000

It looks like the failures are related to the windowsy EOL

Contributor

VladimirNik commented Jan 17, 2014

@adriaanm Thank you for log file. Yes, these problems are caused by windows EOL character. Newline processing and test suite are fixed in #3377.

@gkossakowski gkossakowski referenced this pull request in scala/make-release-notes Mar 5, 2014

Merged

WIP: 2.11.0-RC1 relates notes: breaking changes and deprecations #50

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment