investigate javac compilation error in akka testkit

[adriaanm]

Seems like scala/scala#7966 did break something in akka testkit -- haven't looked beyond copy pasting the error from https://scala-ci.typesafe.com/job/scala-2.13.x-integrate-community-build/2087/consoleText:

akka-testkit/src/main/java/akka/testkit/JavaTestKit.java:150:1: <R>compose(scala.PartialFunction<R,T1>) in scala.runtime.AbstractPartialFunction cannot implement <R>compose(scala.PartialFunction<R,A>) in scala.PartialFunction
  return type scala.PartialFunction<R,R> is not compatible with scala.PartialFunction<R,java.lang.Object>
          new JavaPartialFunction<Object, Object>() {
akka-testkit/src/main/java/akka/testkit/JavaTestKit.java:561:1: <R>compose(scala.PartialFunction<R,T1>) in scala.runtime.AbstractPartialFunction cannot implement <R>compose(scala.PartialFunction<R,A>) in scala.PartialFunction
  return type scala.PartialFunction<R,R> is not compatible with scala.PartialFunction<R,T>
                  new CachingPartialFunction<Object, T>() {

(akka-testkit / Compile / compileIncremental) javac returned non-zero exit code

[adriaanm]

Akka was built from commit c86bd49ab792ecfaaa50c8bfbb5b8677642ed965, which was green on community build run 2082 (using scala 2.13.0-pre-e7d28f4)

[adriaanm]

So, the regression could be caused by any commit in the range scala/scala@e7d28f4...d73f85d

[adriaanm]

Perhaps scala/scala#8037?

[adriaanm]

The latter seems more likely, as it's a javac error, which I can't explain how it could be triggered by scala/scala#7966

[retronym]

I can reproduce locally. I'm whittling down a test case.

[retronym]

package example


trait PartialFunction[-A, +B] extends Function1[A,B] { self =>
  def isDefinedAt(x: A): Boolean
  def compose[R](k: PartialFunction[R, A]): PartialFunction[R, B] = null
}

trait Function1[-T1, +R] extends AnyRef { self =>
  def apply(v1: T1): R
  def compose[A](g: Function1[A, T1]): Function1[A , R] = null
}

abstract class AbstractPartialFunction[-T1, +R] extends Function1[T1, R] with PartialFunction[T1, R] { self =>
  def apply(x: T1): R = ???
}

package example;

public class Client {
    class D extends example.AbstractPartialFunction<String, String> {
        public boolean isDefinedAt(String s) { return false; }
    };
}

[error] /Users/jz/code/ticket-11523/src/main/java/example/Client.java:4:1: <R>compose(example.PartialFunction<R,T1>) in example.AbstractPartialFunction cannot implement <R>compose(example.PartialFunction<R,A>) in example.PartialFunction
[error]   return type example.PartialFunction<R,R> is not compatible with example.PartialFunction<R,java.lang.String>
[error]     class D extends example.AbstractPartialFunction<String, String> {

This went from green to red with scala/scala#8037.

But here's the interesting part: it is also red in 2.12.x.

So the actual point of regression is the library changes in scala/scala@5050915 that introduces an overload of compose into PartialFunction. A later commit did the same with andThen.

The unanswered question is: why does javac reject this? Is there an error with our generic signatures here? Next step of the investigation is to construct an analogous hierarchy in Java and compare the signatures.

[retronym]

Well this is turning into a right rolicking romp.

$ jardiff -c $(ls target/scala-2.12/classes/example/{AbstractPartialFunction,PartialFunction,Function1}.class | paste -d ':' -s -)

+++ Function1.class.asm
// class version 52.0 (52)
// access flags 0x601
// signature <T1:Ljava/lang/Object;R:Ljava/lang/Object;>Ljava/lang/Object;
// declaration: <T1, R>
public abstract interface example/Function1 {


  // access flags 0x9
  public static $init$(Lexample/Function1;)V
    // parameter final synthetic  $this

  // access flags 0x401
  // signature (TT1;)TR;
  // declaration: R (T1)
  public abstract apply(Ljava/lang/Object;)Ljava/lang/Object;
    // parameter final  v1

  // access flags 0x1
  // signature <A:Ljava/lang/Object;>(Lexample/Function1<TA;TT1;>;)Lexample/Function1<TA;TR;>;
  // declaration: example.Function1<A, R> <A>(example.Function1<A, T1>)
  public default compose(Lexample/Function1;)Lexample/Function1;
    // parameter final  g

  // access flags 0x1009
  public static synthetic compose$(Lexample/Function1;Lexample/Function1;)Lexample/Function1;
    // parameter final synthetic  $this
    // parameter final  g
}

+++ AbstractPartialFunction.class.scalap
package example
abstract class AbstractPartialFunction[-T1, +R] extends scala.AnyRef with example.Function1[T1, R] with example.PartialFunction[T1, R] {
  def this() = { /* compiled code */ }
  def apply(x: T1): R = { /* compiled code */ }
}

+++ PartialFunction.class.scalap
package example
trait PartialFunction[-A, +B] extends scala.AnyRef with example.Function1[A, B] {
  def $init$(): scala.Unit = { /* compiled code */ }
  def isDefinedAt(x: A): scala.Boolean
  def compose[R](k: example.PartialFunction[R, A]): example.PartialFunction[R, B] = { /* compiled code */ }
}

+++ PartialFunction.class.asm
// class version 52.0 (52)
// access flags 0x601
// signature <A:Ljava/lang/Object;B:Ljava/lang/Object;>Ljava/lang/Object;Lexample/Function1<TA;TB;>;
// declaration: <A, B> extends example.Function1<A, B>
public abstract interface example/PartialFunction implements example/Function1  {


  // access flags 0x9
  public static $init$(Lexample/PartialFunction;)V
    // parameter final synthetic  $this

  // access flags 0x1
  // signature <R:Ljava/lang/Object;>(Lexample/PartialFunction<TR;TA;>;)Lexample/PartialFunction<TR;TB;>;
  // declaration: example.PartialFunction<R, B> <R>(example.PartialFunction<R, A>)
  public default compose(Lexample/PartialFunction;)Lexample/PartialFunction;
    // parameter final  k

  // access flags 0x1009
  public static synthetic compose$(Lexample/PartialFunction;Lexample/PartialFunction;)Lexample/PartialFunction;
    // parameter final synthetic  $this
    // parameter final  k

  // access flags 0x401
  // signature (TA;)Z
  // declaration: boolean (A)
  public abstract isDefinedAt(Ljava/lang/Object;)Z
    // parameter final  x
}

+++ Function1.class.scalap
package example
trait Function1[-T1, +R] extends scala.AnyRef {
  def $init$(): scala.Unit = { /* compiled code */ }
  def apply(v1: T1): R
  def compose[A](g: example.Function1[A, T1]): example.Function1[A, R] = { /* compiled code */ }
}

+++ AbstractPartialFunction.class.asm
// class version 52.0 (52)
// access flags 0x421
// signature <T1:Ljava/lang/Object;R:Ljava/lang/Object;>Ljava/lang/Object;Lexample/PartialFunction<TT1;TR;>;
// declaration: <T1, R> implements example.PartialFunction<T1, R>
public abstract class example/AbstractPartialFunction implements example/PartialFunction  {


  // access flags 0x1
  // signature ()V
  // declaration: void ()
  public <init>()V

  // access flags 0x1
  // signature (TT1;)TR;
  // declaration: R (T1)
  public apply(Ljava/lang/Object;)Ljava/lang/Object;
    // parameter final  x

  // access flags 0x1
  // signature <R:Ljava/lang/Object;>(Lexample/PartialFunction<TR;TT1;>;)Lexample/PartialFunction<TR;TR;>;
  // declaration: example.PartialFunction<R, R> <R>(example.PartialFunction<R, T1>)
  public compose(Lexample/PartialFunction;)Lexample/PartialFunction;
    // parameter final  k

  // access flags 0x1
  // signature <A:Ljava/lang/Object;>(Lexample/Function1<TA;TT1;>;)Lexample/Function1<TA;TR;>;
  // declaration: example.Function1<A, R> <A>(example.Function1<A, T1>)
  public compose(Lexample/Function1;)Lexample/Function1;
    // parameter final  g
}

Let's focus on AbstarctPartialFunction.compose, the mixin forwarder:

  // signature <R:Ljava/lang/Object;>(Lexample/PartialFunction<TR;TT1;>;)Lexample/PartialFunction<TR;TR;>;
  // declaration: example.PartialFunction<R, R> <R>(example.PartialFunction<R, T1>)
  public compose(Lexample/PartialFunction;)Lexample/PartialFunction;

The return type should actually be PartialFunction<R(method type param, R(class type param)>, but there is no way to express that in the language of generic signatures, as there is no way to express it in Java source code.

We can cheat our way out of this by renaming, say, the type parameter of PartialFunction.compose from R to R1.

What can we do to protect folks from this problem? We could detect this category of unrepresentable generic signature and omit the signature and emit a warning.

It's so sad that we've had to bring back generic signatures for mixin forwarders. 😢

[adriaanm]

Wow.

[retronym]

Another possibility is for the compiler to rename the type parameter in the mixin forwarder. We’d do this selectively to avoid the conflict, as it shows up in type error messages.