Mirror.Sum's MirroredElemLabels unable to distinguish elements

[dwijnand]

Minimized code

sealed trait A

object X { case class Foo(n: Int) extends A }
object Y { case class Foo(n: Int) extends A }

Output

scala> import scala.compiletime._, scala.deriving._

scala> val m = summon[Mirror.Of[A]]
val m:
  (
    deriving.Mirror{
      MirroredType = A; MirroredMonoType = A; MirroredElemTypes <: Tuple
    }
   &
    scala.deriving.Mirror.Sum{
      MirroredMonoType = A; MirroredType = A; MirroredLabel = "A"
    }
  ){MirroredElemTypes = (X.Foo, Y.Foo); MirroredElemLabels = ("Foo", "Foo")} = anon$1@62a41279

Expectation

Due to the two leaves having the same simple name, it's impossible to distinguish them by the labels given.

[bishabosha]

in which case is it necessary to use the labels to distinguish cases, when the MirroredElemTypes or ordinal method can be used instead

[dwijnand]

When deserialising a sum type: you don't have a value to start with so you can't use ordinal. So instead you dispatch on the label.

[bishabosha]

In this situation I would suggest that the result of ordinal should be used as part of the payload, but again this brings up the issue of ordinal not being suitable for versioning your serialisation format to allow deprecation and removal of cases - #10240

[bishabosha]

perhaps we can add a VersionId type to Mirror that can be plugged into with an annotation

[AugustNagro]

@bishabosha I'm a little confused.. if you are given an ordinal and deserializing an enum, how can you materialize the value with Mirror.SumOf?

You can get the type, but then do what?

[bishabosha]

if you are given an ordinal and deserializing an enum, how can you materialize the value with Mirror.SumOf?

You can get the type, but then do what?

So I would say a way to do this in a macro is to first summon inline a Mirror.SumOf for your sum type, then you can enumerate the types of cases by inspecting the type member MirroredElemTypes on that mirror. Then in the same macro, for each case: inline summon a Mirror.ProductOf, which is stored in a Map using the ordinal of that case (i.e. the 0-based index in the MirroredElemTypes tuple), then at runtime you lookup the Mirror by the ordinal, construct a value that conforms to Product with the deserialised elements, (such as Tuple.fromArray, or EmptyTuple for a singleton value/no-param case class) and call fromProduct on the mirror with that Product.

In the case where you know the deserialiser is for an enum, you can optimise lookup for singleton cases by delegating to fromOrdinal on the companion of the enum, but you do need to use the quoted.Quotes reflection API to check if a type is an enum and to fetch companions.

[AugustNagro]

Awesome, thanks a lot! I've got the basics working now without macros, just the derivation tools.

[dwijnand]

I ended up building something like this:

  inline def typeName[L, A]: String = config.typeNaming(summonFullLabel[L, A](summonLabel[L]))

  inline def summonFullLabel[L, A](inline label: L): String = ${summonFullLabelImpl[L, A]('{label}) }

  import scala.quoted._
  private def summonFullLabelImpl[L, A: Type](labelExpr: Expr[L])(using Quotes): Expr[String] = {
    import quotes.reflect._
    val label = labelExpr.asInstanceOf[Expr[String]].valueOrError
    val fqcn  = TypeRepr.of[A].typeSymbol.children.find(_.name == label) match {
      case Some(child) => child.owner.fullName.stripSuffix("$") + "." + child.name
      case None        => label
    }
    Expr(fqcn)
  }

The full diff is in playframework/play-json#572.

[dwijnand]

I got lost in Jamie's explanation after "inline summon a Mirror.ProductOf": is that a pre-existing map and what are the key and value types? I can't tell if I'm using the summoned mirror to find the ordinal or the ordinal to find the mirror... Code would probably clear it all up.

I noticed in your implementation, @AugustNagro, you're not dispatching on the label or the ordinal, you just try each deserialiser until one works:

  private inline def adtSumFromNative[T, Label, Mets <: Tuple](nativeJs: js.Any): T =
    inline erasedValue[Mets] match
      case _: EmptyTuple => throw IllegalArgumentException(
        "Cannot decode " + constString[Label] + " with " + JSON.stringify(nativeJs))
      case _: (met *: metsTail) =>
        try summonInline[NativeConverter[met]].asInstanceOf[NativeConverter[T]].fromNative(nativeJs)
        catch _ => adtSumFromNative[T, Label, metsTail](nativeJs)

As I want the behaviour to match the Scala 2 version of Play JSON, I chose not to go with that implementation.

[bishabosha]

I got lost in Jamie's explanation after "inline summon a Mirror.ProductOf": is that a pre-existing map and what are the key and value types? I can't tell if I'm using the summoned mirror to find the ordinal or the ordinal to find the mirror... Code would probably clear it all up.

This was the idea for the eventual generated code, which is over simplified and uses an imaginary json library.

enum Opt[+T] {
  case Sm(t: T)
  case Nn
}

object Opt {
  given derived$Decoder[T](using Decoder[T]): Decoder[Opt[T]] = new {
    private val mirrors: Map[Int, deriving.Mirror.Product] = Map(0 -> Opt.Sm, 1 -> Opt.Nn)

    def decode(json: Json): Opt[T] = {
      val ord = json("_ordinal").asInt
      val fields = Tuple.fromArray(json.drop("_ordinal").values.toArray) // dropping extra steps to decode the fields
      mirrors(ord).fromProduct(fields).asInstanceOf[Opt[T]]
    }

  }
}

[dwijnand]

Thanks, Jamie. But sadly I can't impose a change in payload.

[AugustNagro]

@dwijnand

At first I wanted to use ordinal but I eventually realized there are serious drawbacks of doing so.

1. If a user re-orders the cases or adds/removes from anywhere but the end, then all clients break.
2. Compatibility with other JSON libraries is confusing. Jackson for instance serializes Java enums with their String type names since Java enums cannot be ADTs. And they use @type property for polymorphic deserialization. This point may only matter for me since my library is Scala.js-only.

So, I handle two cases.

The first is the easiest. When I'm deriving a type with given Mirror.SumOf, I check to see if every element is a singleton:

  /**
   * A singleton is a Product with no parameter elements
   */
  private inline def isSingleton[T]: Boolean = summonFrom[T]:
    case product: Mirror.ProductOf[T] =>
      inline erasedValue[product.MirroredElemTypes] match
        case _: EmptyTuple => true
        case _ => false
    case _ => false

Simple enums have singleton elements:

enum Color(val rgb: Int):
   case Red   extends Color(0xFF0000)
   case Green extends Color(0x00FF00)
   case Blue  extends Color(0x0000FF)

But not ADT enums:

enum Opt[+T] {
  case Sm(t: T)
  case Nn
}

If every element is a Singleton, I serialize and deserialize using the ElemLabels. If not, then I go to the ADT case that you linked.

When I was originally trying to use the ordinal for the ADT Sum case, I would create a native converter like this:

new NativeConverter[T]
  val childConverters: js.Array[NativeConverter[T]] = ??? // call method that recursively builds a js.Array

  def fromNative(nativeJs: js.Any): T =
    val ordinal: Int = nativeJs.asInstanceOf[js.Dynamic].ordinal.asInstanceOf[Int]
    childConverters(ordinal).fromNative(nativeJs)

  extension (t: T) def toNative: js.Any = ??? // use mirror.ordinal(t) to select from childConverters

The implementation of the recursive method just iterated through the ElemTypes tuple, summonInline a NativeConverter for every type, and pushed into the result array (js.Array has [amortized?] O(1) push)

[AugustNagro]

I should also add that when you derive on a generic class like Opt[+T], a new instance is created every time the typeclass is summoned. So try-catch may be faster than repeatedly building a map of converters and using ordinal key. Although I haven't benchmarked. Exceptions are expensive though.