New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Status of support for macros in incremental compiler #1729

Open
Duhemm opened this Issue Nov 13, 2014 · 2 comments

Comments

Projects
None yet
3 participants
@Duhemm
Contributor

Duhemm commented Nov 13, 2014

This is a meta issue that gives an overview of the current state of the support for macro-enabled programs in the incremental compiler.

Support for macros

  • Record dependencies on macro arguments (#1163)
  • Disable macro heuristic for inherited macros (#1142, #1202)
  • Hooking Macrotracker in sbt (#1778, discussion, plugin)
  • Recompile macro clients if any transitive dependency of the macro is updated ( #1836, related issue : #399, discussion)
  • Allow macro clients to depend on auxiliary (non-scala) files (#1757, discussion)

Falicities that help to support macros

These facilities make it easier to extend the kind of dependency relations that sbt knows of, and thus make it easier to implement better support for macros (or anything else that requires adding a new dependency kind).

  • Abstract over dependency kind in Analysis and Relations (#1340)
  • Don't hardcode existing relations in TextAnalysisFormats (#1572)
  • Abstract over dependency kind in Compile (#1736)

Some more words on...

  • Macrotracker is a compiler plugin that registers calls to the reflection API and attaches the list of inspected ("touched") symbols to the expanded macros. These symbols can then be extracted by sbt and registered as dependencies of the macro client.
  • I have a prototype of a solution for the problem of transitive dependencies of macros. It works great when the macro provider has no external dependencies. I have a solution to this problem, but I'm afraid this would hurt performance.
  • Macro providers can add an attachment to the expanded macro to tell sbt that the macro client depends on an auxiliary file by giving its path. An implementation of this proposition can be found here. Example of project that uses this feature

Collecting the dependencies of a macro

Consider the following macro :

object Provider {
  def testMacro(arg: String): String = macro testMacroImpl
  def testMacroImpl(c: Context)(arg: c.Tree): c.Tree = {
    import c.universe._
    val foo = Foo.bar
    val file = scala.io.Source.fromFile("/some/file").mkString
    val result = weakTypeOf[Bar].members.sorted.toString + arg.toString + foo + file
    q"$result"
  }
}

and its application :

object Client extends App {
  println(Provider.testMacro(Fizz.buzz))
}

After compilation, the body of object Client will (roughly) be equivalent to println("List(constructor Bar, ...").

However, many dependencies between object Client and other components exist even though these dependencies cannot be observed at AST level after the expansion of the macro :

object Client depends on...

  • Fizz.buzz because it is an argument of the macro call (fixed, #1163)
  • Foo.bar and on everything that Foo.bar depends on (and so on) (not fixed, see #399).
  • Bar because Bar is inspected using reflection. This is the kind of problem that could be solved using macro tracker.
  • file /some/file because if its content changes, then the expansion of the macro should be different.

/cc @xeno-by @gkossakowski

@gkossakowski

This comment has been minimized.

Show comment
Hide comment
@gkossakowski

gkossakowski Nov 13, 2014

Contributor

Hi @Duhemm!

Excellent overview! Thanks for collecting all references. Would you mind adding a paragraph (at the end) describing the motivation behind this work? It's probably the easiest to describe a simple macro and explain how dependencies coming from a macro are opaque to incremental compiler and why this is a problem. You don't need to discuss all facets of improved macro support, though. Just one simple example should be enough.

Contributor

gkossakowski commented Nov 13, 2014

Hi @Duhemm!

Excellent overview! Thanks for collecting all references. Would you mind adding a paragraph (at the end) describing the motivation behind this work? It's probably the easiest to describe a simple macro and explain how dependencies coming from a macro are opaque to incremental compiler and why this is a problem. You don't need to discuss all facets of improved macro support, though. Just one simple example should be enough.

@Duhemm

This comment has been minimized.

Show comment
Hide comment
@Duhemm

Duhemm Nov 13, 2014

Contributor

Hi @gkossakowski ! Thank you for the suggestion. I've just added an example that covers all problems that I mentioned previously in the post. Do you have any more comments ?

Contributor

Duhemm commented Nov 13, 2014

Hi @gkossakowski ! Thank you for the suggestion. I've just added an example that covers all problems that I mentioned previously in the post. Do you have any more comments ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment