[2.13.9] productIterator change to case classes breaks bincompat if the class is also a value class

[adamw]

MiMA reports the following error when trying to update to 2.13.9, for value classes:

[error] core: Failed binary compatibility check against com.softwaremill.sttp.model:core_2.13:1.5.2! Found 2 potential problems
[error]  * extension static method productIterator$extension(java.lang.String)scala.collection.Iterator in class sttp.model.Method does not have a correspondent in current version
[error]    filter with: ProblemFilters.exclude[DirectMissingMethodProblem]("sttp.model.Method.productIterator$extension")
[error]  * extension method productIterator$extension(java.lang.String)scala.collection.Iterator in object sttp.model.Method does not have a correspondent in current version
[error]    filter with: ProblemFilters.exclude[DirectMissingMethodProblem]("sttp.model.Method.productIterator$extension")

this is for a value case class:

case class Method(method: String) extends AnyVal

is this something that can be safely ignored?

[som-snytt]

I think it is not safe.

Reported at lightbend-labs/mima#723

[adamw]

Oh thanks! I'll close this one as it is clearly a duplicate :)

[som-snytt]

I'll probably need a ticket over here...

[adamw]

Ah sure, sorry, reopening then ;)

[som-snytt]

➜ scalac -d /tmp value.scala
➜ scalac -d /tmp -cp /tmp valuation.scala
➜ scala -cp /tmp Test
hello, world
➜ skalac -d /tmp value.scala  # recompile the case class
➜ scala -cp /tmp Test
Exception in thread "main" java.lang.NoSuchMethodError: 'scala.collection.Iterator Value$.productIterator$extension(java.lang.String)'
        at C.f(valuation.scala:4)

where scalac is 2.13.8 and skalac is 2.13.9 and

➜ cat value.scala

case class Value(s: String) extends AnyVal
➜ cat valuation.scala

class C {
  def f() =
    Value("hello, world").productIterator.foreach(println)
}

object Test extends App {
  new C().f()
}

[SethTisue]

The PR in question is scala/scala#10002 .

In addition figuring out how to address the issue itself, we should also give a little thought to whether our overall testing regimen could have or should have caught this.

(As Som noted on the MiMa ticket, apparently the Scala standard library doesn't have any case classes which are also value classes — if it did, this issue would have shown up.)

[som-snytt]

For convenience, copying my comment

A case class extending AnyVal had product methods:

  public static java.lang.String productElementName$extension(java.lang.String, int);
  public static scala.collection.Iterator<java.lang.Object> productIterator$extension(java.lang.String);
  public static java.lang.Object productElement$extension(java.lang.String, int);
  public static int productArity$extension(java.lang.String);
  public static java.lang.String productPrefix$extension(java.lang.String);
  public scala.collection.Iterator<java.lang.String> productElementNames();
  public java.lang.String productPrefix();
  public int productArity();
  public java.lang.Object productElement(int);
  public scala.collection.Iterator<java.lang.Object> productIterator();
  public java.lang.String productElementName(int);

where productElementNames lacks an extension method and would therefore always box.

Now productIterator also lacks an $extension method for some reason. (Presumably, the reason is that methods inherited from a trait don't get the value class treatment.)

Product extends Any is a universal trait.

[som-snytt]

Worth noting that we had considered binary compatibility in the PR for case classes, but value classes did not appear on the radar.

Sometimes a bytecode diff is performed to witness any changes in output; it would be a bit much to attempt that on CB scala; but one could imagine a modest set of classes in test to validate diffs in bytecode (not only for compatibility).

[som-snytt]

To confirm, the SIP specifically excludes inherited members. https://docs.scala-lang.org/sips/value-classes.html

It is cold comfort that Scala 3 does not generate productIterator for case classes.

[mkurz]

If this is getting fixed, how long until Scala 2.13.10?

[lrytz]

I think we should do 2.13.10 very soon to avoid (too many) releases built with 2.13.9 to make their way into the wild.

Should we keep this ticket open to make it easier to find?

[armanbilge]

I think we should do 2.13.10 very soon to avoid (too many) releases built with 2.13.9 to make their way into the wild.

Can we trust MiMa to catch all instances of this? Or should we avoid releasing projects on 2.13.9 altogether?

[lrytz]

If you're using MiMa you should be safe - modulo unknowns, of course...

[SethTisue]

I think we should do 2.13.10 very soon

There's now a thread to discuss release contents and timing: https://contributors.scala-lang.org/t/scala-2-13-10-release-planning/5943

[SethTisue]

Should we keep this ticket open to make it easier to find?

Closing now since we'll be publishing 2.13.10 shortly, and since we've done a bunch of publicity about the regression in various venues (updated GitHub release notes, Discourse, Twitter, etc)