lambda expressions bootstrap method error

[qiangxu1996]

Hi,

I was trying the following test code but got errors while running the instrumented code or converting it to dalvik bytecode using d8.

import java.util.function.Supplier;

public class SupplierTest {
	public static void main(String[] args) {
		Supplier<double[]> supplier = () -> new double[3];
		System.out.println(supplier.get());
	}
}

The output of java is pasted below.

$ JAVA_HOME=jre-inst jre-inst/bin/java -Xbootclasspath/a:Phosphor-0.0.4-SNAPSHOT.jar SupplierTest
Exception in thread "main" java.lang.BootstrapMethodError: java.lang.NoSuchMethodError: SupplierTest.lambda$main$0$$PHOSPHORTAGGED(Ledu/columbia/cs/psl/phosphor/struct/LazyDoubleArrayIntTags;)Ledu/columbia/cs/psl/phosphor/struct/LazyDoubleArrayIntTags;
	at SupplierTest.main(SupplierTest.java:5)
Caused by: java.lang.NoSuchMethodError: SupplierTest.lambda$main$0$$PHOSPHORTAGGED(Ledu/columbia/cs/psl/phosphor/struct/LazyDoubleArrayIntTags;)Ledu/columbia/cs/psl/phosphor/struct/LazyDoubleArrayIntTags;
	at java.lang.invoke.MethodHandleNatives.resolve(Native Method)
	at java.lang.invoke.MemberName$Factory.resolve$$PHOSPHORTAGGED(MemberName.java:975)
	at java.lang.invoke.MemberName$Factory.resolveOrFail$$PHOSPHORTAGGED(MemberName.java:1000)
	at java.lang.invoke.MemberName$Factory.resolveOrFail(MemberName.java)
	at java.lang.invoke.MethodHandles$Lookup.resolveOrFail(MethodHandles.java:1394)
	at java.lang.invoke.MethodHandles$Lookup.linkMethodHandleConstant(MethodHandles.java:1750)
	at java.lang.invoke.MethodHandleNatives.linkMethodHandleConstant(MethodHandleNatives.java:477)
	... 1 more

The output of d8 is also here.

$ java -jar d8.jar --min-api 28 SupplierTest.class
Compilation failed with an internal error.
com.android.tools.r8.errors.Unreachable: Enforced and erased signatures are inconsistent in CallSite: { Name: get, Proto: Proto L java.util.function.Supplier , MethodHandle: {INVOKE_STATIC, java.lang.invoke.CallSite java.lang.invoke.LambdaMetafactory.metafactory(java.lang.invoke.MethodHandles$Lookup, java.lang.String, java.lang.invoke.MethodType, java.lang.invoke.MethodType, java.lang.invoke.MethodHandle, java.lang.invoke.MethodType)}, Args: Item 21 Proto L java.lang.Object , Item 22 MethodHandle: {INVOKE_STATIC, edu.columbia.cs.psl.phosphor.struct.LazyDoubleArrayIntTags SupplierTest.lambda$main$0$$PHOSPHORTAGGED(edu.columbia.cs.psl.phosphor.struct.LazyDoubleArrayIntTags)}, Item 21 Proto LL edu.columbia.cs.psl.phosphor.struct.LazyDoubleArrayIntTags  edu.columbia.cs.psl.phosphor.struct.LazyDoubleArrayIntTags}
	at com.android.tools.r8.ir.desugar.LambdaDescriptor.infer(LambdaDescriptor.java:268)
	at com.android.tools.r8.ir.desugar.LambdaRewriter.inferLambdaDescriptor(LambdaRewriter.java:221)
	at com.android.tools.r8.ir.desugar.LambdaRewriter.desugarLambdas(LambdaRewriter.java:117)
	at com.android.tools.r8.ir.conversion.IRConverter.rewriteCode(IRConverter.java:834)
	at com.android.tools.r8.ir.conversion.IRConverter.convertMethodToDex(IRConverter.java:450)
	at com.android.tools.r8.ir.conversion.IRConverter.lambda$convertMethodsToDex$3(IRConverter.java:438)
	at com.android.tools.r8.graph.DexClass.forEachMethod(DexClass.java:134)
	at com.android.tools.r8.ir.conversion.IRConverter.convertMethodsToDex(IRConverter.java:436)
	at com.android.tools.r8.ir.conversion.IRConverter.lambda$convertClassesToDex$2(IRConverter.java:423)
	at java.util.concurrent.ForkJoinTask$AdaptedRunnableAction.exec(ForkJoinTask.java:1386)
	at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
	at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
	at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
	at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157)

I am using OpenJDK 1.8.0_152 and d8 1.3.49.

I looked into the source code of Phosphor and found the third argument of the bootstrap method for the lambda expression causes the problem.

Before instrumentation, the invokedynamic instruction for the lambda expression looks like this:

    INVOKEDYNAMIC get()Ljava/util/function/Supplier; [
      // handle kind 0x6 : INVOKESTATIC
      java/lang/invoke/LambdaMetafactory.metafactory(Ljava/lang/invoke/MethodHandles$Lookup;Ljava/lang/String;Ljava/lang/invoke/MethodType;Ljava/lang/invoke/MethodType;Ljava/lang/invoke/MethodHandle;Ljava/lang/invoke/MethodType;)Ljava/lang/invoke/CallSite;
      // arguments:
      ()Ljava/lang/Object;,
      // handle kind 0x6 : INVOKESTATIC
      SupplierTest.lambda$main$0()[D,
      ()[D
    ]

That instruction is converted as below after instrumentation:

    INVOKEDYNAMIC get()Ljava/util/function/Supplier; [
      // handle kind 0x6 : INVOKESTATIC
      java/lang/invoke/LambdaMetafactory.metafactory(Ljava/lang/invoke/MethodHandles$Lookup;Ljava/lang/String;Ljava/lang/invoke/MethodType;Ljava/lang/invoke/MethodType;Ljava/lang/invoke/MethodHandle;Ljava/lang/invoke/MethodType;)Ljava/lang/invoke/CallSite;
      // arguments:
      ()Ljava/lang/Object;,
      // handle kind 0x6 : INVOKESTATIC
      SupplierTest.lambda$main$0$$PHOSPHORTAGGED(Ledu/columbia/cs/psl/phosphor/struct/LazyDoubleArrayIntTags;)Ledu/columbia/cs/psl/phosphor/struct/LazyDoubleArrayIntTags;,
      (Ledu/columbia/cs/psl/phosphor/struct/LazyDoubleArrayIntTags;)Ledu/columbia/cs/psl/phosphor/struct/LazyDoubleArrayIntTags;
    ]

The extra parameter (type Ledu/columbia/cs/psl/phosphor/struct/LazyDoubleArrayIntTags) in the second line from bottom causes the signature inconsistency. That parameter is added in TaintUtils.java:799. But I am not sure why it is needed. Please have a look at it.

Thanks,
Qiang

[jon-bell]

From what you've pasted you are not adding the java agent when running - can you try again with it?

[qiangxu1996]

Thank you for getting back quickly. The result is the same. The classes are all pre-instrumented.

[jon-bell]

Thanks for the minimum working example - I was able to reproduce this easily. This is now patched in 9a56248, and I've added some new tests to the suite.

FYI - my prior warnings and disclaimers about using Phosphor on Android (especially post-ART) all still apply - you will start to hit much more complex and Android-specific issues, and I will not be able to support them (it will be up to you).

[qiangxu1996]

Thank you for the quick fix. I will try to fix those Android specific ones.