Scala 2.11.12 quasi-quote interpolation "losing" trees in nested macros invocations

[edmondop]

Introduction

In Avro4s, the SchemaFor[A] typeclass is responsible to generate an Avro schema for a Scala case class, while the ToRecord[A] generates a GenericRecord for a Scala case class. Therefore, ToRecord[A] makes typically usage of SchemaFor[A]

Scenario

As we are trying to extend the library to support sealed hierarchy of case classes, we are encountering an unexpected behaviour of Scalac 2.11.12. Take the following two case classes:

      case class Ingredient(name: String, sugar: Double, fat: Double)
      case class Pizza(name: String, ingredients: Seq[Ingredient], vegetarian: Boolean, vegan: Boolean, calories: Int)

When we materialize the schema for both by invoking SchemaFor.materialize[Ingredient] and SchemaFor.materialize[Pizza] everything compiles fine and the materialization for the Pizza will recursively invoke the materialization for the Ingredient. When we try to materialize the ToRecord though, the compiler will "forget" to include an invocation of another macro inside the materialization of SchemaFor and will fail with the error:

[error] symbol value inst$macro$39#81224 does not exist in com.sksamuel.avro4s.examples.Examples$$anonfun$1$$anonfun$apply$mcV$sp$1$$anon$1$anon$lazy$macro$75$1$$anon$2$$anon$8$$anonfun$7$$anonfun$apply$15.apply, which contains locals . 
[error] Method code: final def apply(): com#9.sksamuel#8242.avro4s#8246.ToSchema$StringToSchema#55746.type = inst$macro$39
[trace] Stack trace suppressed: run last avro4s-core/test:compileIncremental for the full output.

Macro details

The ToRecord.materialize[A] macro tries to resolve an implicit SchemaFor[A] from the context, which will turn into an invocation of SchemaFor.materialize[A] if the implicit is not found. At the gist https://gist.github.com/edmondo1984/f41f34661ed3abee08d1d1f3e028b4ad the macro are available as well as the tree produced by the macros.

At line 115 of SchemaFor.scala, the following snippet is interpolated:

  q"""{ _root_.com.sksamuel.avro4s.SchemaFor.fieldBuilder[$sig]($fieldName, Seq(..$annos), null, $defaultNamespace) }"""

the function is located here (the whole code has been omitted from the gist)

  def fieldBuilder[T](name: String, annos: Seq[Anno], default: Any, parentNamespace: String)
                     (implicit toSchema: Lazy[ToSchema[T]]): Schema.Field = {
    fieldBuilder(name, annos, toSchema.value.apply(), default, parentNamespace)
  }

When the invocation to SchemaFor.materialize[Ingredient] occurs inside an invocation of Record.materialize[Ingredient] the produced tree is perfectly valid and no compilation error occurs (lines 26-35 of ToRecordIngredient.scala)

     collection.this.Seq.apply[org.apache.avro.Schema.Field](com.sksamuel.avro4s.SchemaFor.fieldBuilder[String]("name", collection.this.Seq.apply[Nothing](), null, "com.sksamuel.avro4s.examples")({
            val inst$macro$6: com.sksamuel.avro4s.ToSchema.StringToSchema.type = avro4s.this.ToSchema.StringToSchema.asInstanceOf[com.sksamuel.avro4s.ToSchema.StringToSchema.type];
            shapeless.Lazy.apply[com.sksamuel.avro4s.ToSchema.StringToSchema.type](inst$macro$6)
          }), com.sksamuel.avro4s.SchemaFor.fieldBuilder[Double]("sugar", collection.this.Seq.apply[Nothing](), null, "com.sksamuel.avro4s.examples")({
            val inst$macro$8: com.sksamuel.avro4s.ToSchema.DoubleToSchema.type = avro4s.this.ToSchema.DoubleToSchema.asInstanceOf[com.sksamuel.avro4s.ToSchema.DoubleToSchema.type];
            shapeless.Lazy.apply[com.sksamuel.avro4s.ToSchema.DoubleToSchema.type](inst$macro$8)
          }), com.sksamuel.avro4s.SchemaFor.fieldBuilder[Double]("fat", collection.this.Seq.apply[Nothing](), null, "com.sksamuel.avro4s.examples")({
            val inst$macro$10: com.sksamuel.avro4s.ToSchema.DoubleToSchema.type = avro4s.this.ToSchema.DoubleToSchema.asInstanceOf[com.sksamuel.avro4s.ToSchema.DoubleToSchema.type];
            shapeless.Lazy.apply[com.sksamuel.avro4s.ToSchema.DoubleToSchema.type](inst$macro$10)
          }))

I.e. for each interpolation of the previous snippet, a val inst$macro$N is created and this is passed to shapeless.Lazy.apply.

Unexpected behaviour

When invoking Record.materialize[Pizza] this would trigger a call to SchemaFor.materialize[Pizza] which in turn will invoke a SchemaFor.materialize[Ingredient]. The tree produced by this expansion is very similar to the previous one (line 175 of ToRecordPizza.scala)

collection.this.Seq.apply[org.apache.avro.Schema.Field](com.sksamuel.avro4s.SchemaFor.fieldBuilder[String]("name", collection.this.Seq.apply[Nothing](), null, "com.sksamuel.avro4s.examples")(shapeless.Lazy.apply[com.sksamuel.avro4s.ToSchema.StringToSchema.type](inst$macro$39)), com.sksamuel.avro4s.SchemaFor.fieldBuilder[Double]("sugar", collection.this.Seq.apply[Nothing](), null, "com.sksamuel.avro4s.examples")(shapeless.Lazy.apply[com.sksamuel.avro4s.ToSchema.DoubleToSchema.type](inst$macro$40)), com.sksamuel.avro4s.SchemaFor.fieldBuilder[Double]("fat", collection.this.Seq.apply[Nothing](), null, "com.sksamuel.avro4s.examples")(shapeless.Lazy.apply[com.sksamuel.avro4s.ToSchema.DoubleToSchema.type](inst$macro$40)))

but in this case there is no explicit creation of val inst$macro$39 , inst$macro$40 and the compilation fails

[error] symbol value inst$macro$39#81224 does not exist in com.sksamuel.avro4s.examples.Examples$$anonfun$1$$anonfun$apply$mcV$sp$1$$anon$1$anon$lazy$macro$75$1$$anon$2$$anon$8$$anonfun$7$$anonfun$apply$15.apply, which contains locals . 
[error] Method code: final def apply(): com#9.sksamuel#8242.avro4s#8246.ToSchema$StringToSchema#55746.type = inst$macro$39

[SethTisue]

closing all quasiquotes tickets; see #10755