Difference in compiler behavior 3.1.2 vs 3.1.3 - implicit resolution / type mismatch

[ALPSMAC]

Compiler version

3.1.2 / 3.1.3

Minimized code

This one's in production code and difficult to provide a minimized example for, so apologies ahead of time...

This issue arises in the context of implicit resolution to find (I'm presuming) the applicable ToEntityMarshaller for the JsArray type.

For the implicit marshaller support I copied into my baseline the PlayJsonSupport trait from here: https://github.com/hseeberger/akka-http-json/blob/master/akka-http-play-json/src/main/scala/de/heikoseeberger/akkahttpplayjson/PlayJsonSupport.scala since it's only one file and the library is not yet available in general for Scala 3.

// In an Akka-Http route using the following library versions: 
//  val playJsonVersion          = "2.10.0-RC6"
//  val akkaHttpVersion          = "10.2.9"
//  val akkaVersion              = "2.6.19"
// All Akka dependencies are cross-compiled w/ `for3Use2_13`

// ...
def generateJsArray: JsArray = { /* ... */ }
//...

// some path routing logic in the Akka-Http server DSL then: 
get {
  complete[JsArray](StatusCodes.OK, generateJsArray)
}

UPDATE: Here's a repo with a self-contained example. Change the scalaVersion in build.sbt to reproduce the issue: https://github.com/ALPSMAC/sample-scala3-compiler-bug/tree/main

Output

Under Scala 3.1.2 this compiles as expected. Under Scala 3.1.3 I get a Type Mismatch Error:

[error] -- [E057] Type Mismatch Error: /home/apolack/work/projects/(elided)/src/main/scala/(elided)/ProductionRESTAPI.scala:268:23 
[error] 268 |                      )
[error]     |                       ^
[error]     |Type argument Any does not conform to upper bound play.api.libs.json.JsValue
[error]     |---------------------------------------------------------------------------
[error]     | Explanation (enabled by `-explain`)
[error]     |- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
[error]     | I tried to show that
[error]     |   Any
[error]     | conforms to
[error]     |   play.api.libs.json.JsValue
[error]     | but the comparison trace ended with `false`:
[error]     |
[error]     |   ==> Any  <:  play.api.libs.json.JsValue
[error]     |     ==> Any  <:  play.api.libs.json.JsValue
[error]     |       ==> Any  <:  play.api.libs.json.JsValue
[error]     |       <== Any  <:  play.api.libs.json.JsValue = false
[error]     |     <== Any  <:  play.api.libs.json.JsValue = false
[error]     |   <== Any  <:  play.api.libs.json.JsValue = false
[error]     |
[error]     | The tests were made under the empty constraint
[error]      ---------------------------------------------------------------------------
[error] Explanation
[error] ===========
[error] I tried to show that
[error]   Any
[error] conforms to
[error]   play.api.libs.json.JsValue
[error] but the comparison trace ended with `false`:
[error] 
[error]   ==> Any  <:  play.api.libs.json.JsValue
[error]     ==> Any  <:  play.api.libs.json.JsValue
[error]       ==> Any  <:  play.api.libs.json.JsValue
[error]       <== Any  <:  play.api.libs.json.JsValue = false
[error]     <== Any  <:  play.api.libs.json.JsValue = false
[error]   <== Any  <:  play.api.libs.json.JsValue = false
[error] 
[error] The tests were made under the empty constraint
[error] one error found

Thinking perhaps passing the JsArray as a type argument explicitly to complete would help I tried that... but either way (with or without the explicit type argument) I still get the same compiler error under 3.1.3, but not under 3.1.2.

Expectation

The code should compile under Scala 3.1.3... or if not, at least consistently not compile under both 3.1.2 and 3.1.3. I wouldn't expect a minor compiler revision to break backwards compatibility in this way.

Thanks

Thanks for all of the great work on Scala 3. I'm looking forward to the ecosystem further evolving and stabilizing.

[SethTisue]

Is the problem also reproducible in 3.2.0-RC3?

[ALPSMAC]

Yes - just confirmed it's an issue in 3.2.0-RC3 as well.

[ALPSMAC]

Not sure if JVM version could have an impact here or not... but for completeness sake:

$ java -version
java version "17.0.3.1" 2022-04-22 LTS
Java(TM) SE Runtime Environment (build 17.0.3.1+2-LTS-6)
Java HotSpot(TM) 64-Bit Server VM (build 17.0.3.1+2-LTS-6, mixed mode, sharing)

[ALPSMAC]

Pushed a repo that replicates the behavior: https://github.com/ALPSMAC/sample-scala3-compiler-bug/tree/main

If you change the scalaVersion in build.sbt from 3.1.2 to 3.1.3 or 3.2.0-RC3 you can cause the compilation error.

Thanks!

[odersky]

It would be good to minimize this further so that it's a self contained example without external references to projects. Without it, I don't see a big chance that this will be addressed.

[ALPSMAC]

Thanks @odersky - would like to but am currently fighting 🔥 s that are limiting my ability to contribute more time to helping to sleuth out the root of the bug or difference in behavior... at least in the very short term.

Happy to return to help find a more self-contained example later, but that might not be for a couple of weeks given my current schedule realities.

[odersky]

OK, thanks! We'll wait then.

[ALPSMAC]

I had a little bit of time just now to play with this... and I haven't been able to reproduce it in a simpler scenario yet. I do know what does not produce the difference in behavior. Namely this example, complex at it is, does seem to type check and compile correctly:

object MinimizedExample {

  object Setup {
    sealed abstract class Marshaller[-A, +B] {
      def toB(a: A): B
    }

    type ToMessageMarshaller[T] = Marshaller[T, Message]

    final case class Message(bytes: Array[Byte])

    final case class RequestContext[T](t: T)

    final case class RouteResult(status: Int, message: Message)

    type AsyncAction[T] = RequestContext[T] => scala.concurrent.Future[RouteResult]

    trait JsValue {
      val rawValue: String
      def toBytes = rawValue.getBytes("utf8")
    }
    case class JsArray(value: scala.collection.IndexedSeq[JsValue] = Array[JsValue]()) extends JsValue{
      override val rawValue: String = value.map{_.rawValue}.mkString(",")
    }
    case class JsString(str: String) extends JsValue{
      override val rawValue: String = str
    }

    trait Writes[A]{
      def write(a: A): JsValue
    }

    implicit def marshaller[A](
      implicit writes: Writes[A],
      printer: JsValue => String = { v => v.rawValue }
    ): ToMessageMarshaller[A] = {
      new Marshaller[A, Message]{
        override def toB(a: A): Message = Message(writes.write(a).toBytes)
      }
    }

    final case class Response(a: String, b: String, c: String){
      def toJson = responseWrites.write(this)
    }

    implicit val responseWrites: Writes[Response] = new Writes[Response]{
      override def write(r: Response): JsValue = JsString(r.a + ":" + r.b + ":" + r.c)
    }

    // Identity - not sure where this gets defined originally but presumably it must for things to have been
    // working in 3.1.2...???
    implicit val writesJsValue: Writes[JsValue] = new Writes[JsValue]{
      override def write(v: JsValue): JsValue = v
    }

    import scala.concurrent.ExecutionContext.Implicits.global

    def complete[T](status: Int, t: => T)(implicit m: ToMessageMarshaller[T]): AsyncAction[T] = {
      { (ctx: RequestContext[T]) =>
        scala.concurrent.Future(RouteResult(
          status, m.toB(t)
        ))
      }
    }
  }

  object Use {
    import Setup._

    def responses: JsArray = JsArray((0 until 10).map{ i =>
      Response(i.toString, "j", "k").toJson
    })

    complete(200, responses)
  }

}

When I have some more time later I'll try to circle back around and see if I can't narrow the scope more... but I'm posting my findings thus-far on the off chance that knowing what is working helps those more qualified than myself in the guts of the Scala compiler narrow the cause.