Integrate macro paradise under `-Ymacro-annotations` #6606

adriaanm · 2018-05-06T18:35:02Z

Importing from scalamacros/paradise, since xeno-by has stepped down
as maintainer. Since the eco-system has come to rely on macro paradise,
we acknowledge the status quo by integrating paradise.

HOWEVER, please note that support for macro annotations remains
EXPERIMENTAL, and support for them will be much more restricted
in Scala 3. We will try to provide a preview of this subset in 2.14.

Most notably, we do not intend to let macro annotations
synthesize code that is visible during synthesis. Concretely,
you'd have to first compile the macros, then the code that
uses them as annotations, and finally the code that needs to
see the code synthesized by the annotation macros. This is
the same workflow as usual with code generation.

The tests are included as a new project (macroAnnot),
which uses the quick compiler as a scalaInstance to compile
the tests, so that we could integrate the test suite more easily.

Finally, note the diff re: List toStream flatMap headOption:
its behavior changed with the new collections.

adriaanm · 2018-05-06T18:35:18Z

/cc @xeno-by

milessabin · 2018-05-06T21:10:02Z

Bravo!

hrhino · 2018-05-06T21:18:43Z

Indeed! 🎉 Is the -Y flag temporary?

adriaanm · 2018-05-07T06:53:07Z

My current thinking is that it will remain under a -Y flag, as much as I would like to get rid of those. The implementation is pretty hairy, and affects the core of type checking in namers. (That's why we kept it in a plugin originally.)

SethTisue · 2018-05-07T06:56:21Z

Agree on the -Y, it's a marker that it remains permanently-experimental-until-Scala-3-at-which-point-things-change.

xeno-by

Thank you for giving macro paradise a new home! I am ultra thrilled to see this happening 😍

xeno-by · 2018-05-07T17:37:30Z

src/compiler/scala/tools/nsc/Global.scala

-    val global: Global.this.type = Global.this
-  } with Analyzer
+  lazy val analyzer =
+    if (settings.YmacroAnnotations) new { val global: Global.this.type = Global.this } with Analyzer with MacroAnnotationNamers


xeno-by · 2018-05-07T17:46:29Z

src/compiler/scala/tools/nsc/typechecker/MacroAnnotationNamers.scala

+
+  override def newNamer(context: Context): Namer = new MacroAnnotationNamer(context)
+
+  private class MacroAnnotationNamer(context: Context) extends Namer(context) {


Can we have this class (as well as the accompanying Typer) publicized? In our SemanticDB compiler plugin, we are hijacking namers and typers to retain some information about tree originals, so we'll need to inherit from this class to be compatible with macro annotations (https://github.com/scalameta/scalameta/blob/v3.7.3/semanticdb/scalac/library/src/main/scala-2.11.11/scala/meta/internal/semanticdb/scalac/SemanticdbAnalyzer.scala#L20).

sure, but I am planning to rework this code for M5

xeno-by · 2018-05-07T17:48:20Z

src/compiler/scala/tools/nsc/typechecker/ContextErrors.scala

+
+      def MacroAnnotationNotExpandedMessage = {
+        "macro annotation could not be expanded " +
+        "(the most common reason for that is that you need to enable the macro paradise plugin; " +


Am I right in thinking that there will no longer be a need in the plugin, and this is just a typo?

thanks, fixed

xeno-by · 2018-05-07T17:51:01Z

src/compiler/scala/tools/nsc/typechecker/MacroAnnotationNamers.scala

+        else NoSymbol
+      }
+      // def isAccessible(cx: Context, sym: Symbol) = if (canDefineMann(cx.owner)) cx.isAccessible(sym, cx.prefix, superAccess = false) else false
+      def isAccessible(cx: Context, sym: Symbol) = true // TODO: sorry, it's 2am, and I can't figure this out


I am excited to see this comment making it to scala/scala :)

Importing from scalamacros/paradise, since xeno-by has stepped down as maintainer. Since the eco-system has come to rely on macro paradise, we acknowledge the status quo by integrating paradise. HOWEVER, please note that support for macro annotations remains EXPERIMENTAL, and support for them will be much more restricted in Scala 3. We will try to provide a preview of this subset in 2.14. Most notably, we do not intend to let macro annotations synthesize code that is visible during synthesis. Concretely, you'd have to first compile the macros, then the code that uses them as annotations, and finally the code the needs to see the code synthesized by the annotation macros. This is the same workflow as usual with code generation. The tests are included as a new project (macroAnnot), which uses the quick compiler as a scalaInstance to compile the tests, so that we could integrate the test suite more easily. Finally, note the diff re: `List toStream flatMap headOption`: its behavior changed with the new collections.

adriaanm · 2018-05-08T13:23:39Z

src/compiler/scala/tools/nsc/typechecker/MacroAnnotationNamers.scala

+              } else None
+            } else None
+
+          annZippers.iterator.flatMap(annz => maybeExpand(annz.annotation, annz.annottee, annz.owner)).nextOption match {


@szeiger this used to be annZippers.toStream.flatMap(annz => maybeExpand(annz.annotation, annz.annottee, annz.owner)).headOption, which fails in macroAnnot/testOnly Multiple

adriaanm · 2018-05-08T13:55:54Z

Since I didn't preserve history, here's the core of the diff: https://gist.github.com/adriaanm/bdaec0588ad6fdf3bbe6ea09d31edf87

produced with

cat plugin/src/main/scala/org/scalamacros/paradise/typechecker/Namers.scala plugin/src/main/scala/org/scalamacros/paradise/typechecker/Expanders.scala > /tmp/para-namers.scala
diff -uw  /tmp/para-namers.scala ../scala/src/compiler/scala/tools/nsc/typechecker/MacroAnnotationNamers.scala

SethTisue · 2018-05-09T16:54:32Z

a 2.13 community build run on this is here: https://scala-ci.typesafe.com/job/scala-2.13.x-integrate-community-build/1095/ . (it works by enabling -Ymacro-annotations globally and removing paradise from libraryDependencies globally)

the run shows:

enabling -Ymacro-annotations everywhere didn't break anything
two paradise-using projects went green: macro-compat (which has a "bundle" macro), minitest (which has a "source location" macro)
all failing projects are failing for collections reasons, not paradise reasons

in my opinion, this is the best we can expect at this stage, and this PR should be merged, with the expectation that it may need refinement later. it is at least minimally functional — the 2.13 community build can't currently tell us more than that. we can find out more, post-M4.

the only way I can see that we could get substantially better feedback on this, right now, would be to backport this PR to 2.12 and run the 2.12 community build on it.

soronpo · 2018-05-09T18:09:22Z

the only way I can see that we could get substantially better feedback on this, right now, would be to backport this PR to 2.12 and run the 2.12 community build on it.

Is it not easier to create a fork of 2.13-M3 and apply this PR on it to test the community build on?

adriaanm · 2018-05-09T18:14:13Z

I do intend to backport this to 2.12, hopefully in time for 2.12.7, but that has lower priority than 2.13 milestone tasks, which will keep me plenty busy for the coming months :-)

SethTisue · 2018-05-09T20:34:03Z

Is it not easier to create a fork of 2.13-M3 and apply this PR on it to test the community build on?

(probably, but even the 2.13-community-build-as-of-2.13.0-M3 didn't have anywhere near as many projects in it as the 2.12 one)

raulraja · 2018-05-16T18:53:21Z

I just want to say thank you! This is huge for Freestyle

Ref scala/scala#6606

scala-jenkins added this to the 2.13.0-M5 milestone May 6, 2018

adriaanm force-pushed the macro-annot branch from 86f63c4 to ead62d5 Compare May 6, 2018 20:09

SethTisue mentioned this pull request May 7, 2018

2.13: move to new collections scala/community-build#710

Closed

olafurpg mentioned this pull request May 7, 2018

SPIKE: Replacements for scalameta-based macros frees-io/freestyle#539

Closed

adriaanm modified the milestones: 2.13.0-M5, 2.13.0-M4 May 7, 2018

xeno-by reviewed May 7, 2018

View reviewed changes

adriaanm force-pushed the macro-annot branch from f3e6b53 to cc63d2a Compare May 8, 2018 09:47

adriaanm force-pushed the macro-annot branch from ccf5073 to 50077e9 Compare May 8, 2018 12:44

adriaanm changed the title ~~wip: bring paradise down to earth~~ Integrate macro paradise under -Ymacro-annotations May 8, 2018

adriaanm requested a review from lrytz May 8, 2018 12:46

adriaanm commented May 8, 2018

View reviewed changes

This comment has been minimized.

Sign in to view

SethTisue mentioned this pull request May 8, 2018

2.13: replace macro paradise dependencies with -Ymacro-annotations scala/community-build#724

Closed

This comment has been minimized.

Sign in to view

lrytz approved these changes May 10, 2018

View reviewed changes

adriaanm merged commit c4adc81 into scala:2.13.x May 10, 2018

szeiger mentioned this pull request May 13, 2018

Stream iterator is not lazy enough scala/bug#10878

Closed

SethTisue added the release-notes worth highlighting in next release notes label May 14, 2018

olafurpg mentioned this pull request May 25, 2018

release for scala 2.12.6 scalameta/scalameta#1515

Closed

eed3si9n added a commit to eed3si9n/minitest that referenced this pull request May 28, 2018

remove macro paradise for 2.13

d201920

Ref scala/scala#6606

eed3si9n mentioned this pull request May 28, 2018

remove macro paradise for 2.13 monix/minitest#14

Merged

juanpedromoreno mentioned this pull request Jun 4, 2018

frees-rpc new Generation higherkindness/mu-scala#290

Closed

xuwei-k mentioned this pull request Jul 1, 2018

Release for 2.13.0-M4 scalamacros/paradise#118

Closed

diesalbla mentioned this pull request Jul 22, 2018

PROPOSAL - Migrate Scalameta Macros typelevel/cats-tagless#6

Closed

SethTisue mentioned this pull request Sep 24, 2018

Status of macro paradise in Scala 2.13 series circe/circe#975

Closed

ebruchez mentioned this pull request Jun 9, 2019

XPath: replace reflective calls with native functions orbeon/orbeon-forms#2214

Closed

L-Lavigne mentioned this pull request Jul 29, 2019

Replace macros with code generation higherkindness/mu-scala#631

Open

denisftw mentioned this pull request Jul 30, 2019

Macro compilation issues with Scala 2.13 scala-hamsters/hamsters#118

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Integrate macro paradise under `-Ymacro-annotations` #6606

Integrate macro paradise under `-Ymacro-annotations` #6606

adriaanm commented May 6, 2018 •

edited

adriaanm commented May 6, 2018

milessabin commented May 6, 2018

hrhino commented May 6, 2018

adriaanm commented May 7, 2018

SethTisue commented May 7, 2018 •

edited

xeno-by left a comment

xeno-by May 7, 2018

xeno-by May 7, 2018

adriaanm May 8, 2018

xeno-by May 7, 2018

adriaanm May 8, 2018

xeno-by May 7, 2018

adriaanm May 8, 2018

This comment has been minimized.

This comment has been minimized.

adriaanm commented May 8, 2018

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

SethTisue commented May 9, 2018 •

edited

soronpo commented May 9, 2018 •

edited

adriaanm commented May 9, 2018

SethTisue commented May 9, 2018

raulraja commented May 16, 2018


		override def newNamer(context: Context): Namer = new MacroAnnotationNamer(context)

		private class MacroAnnotationNamer(context: Context) extends Namer(context) {

Integrate macro paradise under -Ymacro-annotations #6606

Integrate macro paradise under -Ymacro-annotations #6606

Conversation

adriaanm commented May 6, 2018 • edited

adriaanm commented May 6, 2018

milessabin commented May 6, 2018

hrhino commented May 6, 2018

adriaanm commented May 7, 2018

SethTisue commented May 7, 2018 • edited

xeno-by left a comment

Choose a reason for hiding this comment

xeno-by May 7, 2018

Choose a reason for hiding this comment

xeno-by May 7, 2018

Choose a reason for hiding this comment

adriaanm May 8, 2018

Choose a reason for hiding this comment

xeno-by May 7, 2018

Choose a reason for hiding this comment

adriaanm May 8, 2018

Choose a reason for hiding this comment

xeno-by May 7, 2018

Choose a reason for hiding this comment

adriaanm May 8, 2018

Choose a reason for hiding this comment

This comment has been minimized.

This comment has been minimized.

adriaanm commented May 8, 2018

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

SethTisue commented May 9, 2018 • edited

soronpo commented May 9, 2018 • edited

adriaanm commented May 9, 2018

SethTisue commented May 9, 2018

raulraja commented May 16, 2018

Integrate macro paradise under `-Ymacro-annotations` #6606

Integrate macro paradise under `-Ymacro-annotations` #6606

adriaanm commented May 6, 2018 •

edited

SethTisue commented May 7, 2018 •

edited

SethTisue commented May 9, 2018 •

edited

soronpo commented May 9, 2018 •

edited