v1.57.0 fails with Java 8

[bestbeforetoday]

What version of gRPC-Java are you using?

1.57.0

What is your environment?

• Java Temurin version 8.0.372+7
• Ubuntu 22.0.4 LTS

What did you expect to see?

Remote gRPC service calls work.

What did you see instead?

Code that ran perfectly with previous gRPC-Java versions now fails only on Java 8, with the following error:

java.lang.NoSuchMethodError: java.nio.ByteBuffer.clear()Ljava/nio/ByteBuffer;

The same code continues to run fine with Java 11 and Java 17. Failures only occur with Java 8 using v1.57.0 of gRPC-Java. Might this be a symptom of the published code being compiled with a later Java version without the compiler release option being correctly set to 8, and so generating bytecode that refers to a method overload that does not exist in the Java 8 standard library?

Using a blocking stub, this is the error:

java.lang.NoSuchMethodError: java.nio.ByteBuffer.clear()Ljava/nio/ByteBuffer;
	at io.grpc.internal.MessageFramer.writeKnownLengthUncompressed(MessageFramer.java:225)
	at io.grpc.internal.MessageFramer.writeUncompressed(MessageFramer.java:169)
	at io.grpc.internal.MessageFramer.writePayload(MessageFramer.java:142)
	at io.grpc.internal.AbstractStream.writeMessage(AbstractStream.java:66)
	at io.grpc.internal.ForwardingClientStream.writeMessage(ForwardingClientStream.java:37)
	at io.grpc.internal.DelayedStream$6.run(DelayedStream.java:283)
	at io.grpc.internal.DelayedStream.drainPendingCalls(DelayedStream.java:182)
	at io.grpc.internal.DelayedStream.access$100(DelayedStream.java:44)
	at io.grpc.internal.DelayedStream$4.run(DelayedStream.java:148)
	at io.grpc.stub.ClientCalls$ThreadlessExecutor.runQuietly(ClientCalls.java:777)
	at io.grpc.stub.ClientCalls$ThreadlessExecutor.waitAndDrain(ClientCalls.java:760)
	at io.grpc.stub.ClientCalls.blockingUnaryCall(ClientCalls.java:161)

Using a future stub, this is the error:

Exception in thread "grpc-default-executor-0" java.lang.NoSuchMethodError: java.nio.ByteBuffer.clear()Ljava/nio/ByteBuffer;
	at io.grpc.internal.MessageFramer.writeKnownLengthUncompressed(MessageFramer.java:225)
	at io.grpc.internal.MessageFramer.writeUncompressed(MessageFramer.java:169)
	at io.grpc.internal.MessageFramer.writePayload(MessageFramer.java:142)
	at io.grpc.internal.AbstractStream.writeMessage(AbstractStream.java:66)
	at io.grpc.internal.ForwardingClientStream.writeMessage(ForwardingClientStream.java:37)
	at io.grpc.internal.DelayedStream$6.run(DelayedStream.java:283)
	at io.grpc.internal.DelayedStream.drainPendingCalls(DelayedStream.java:182)
	at io.grpc.internal.DelayedStream.access$100(DelayedStream.java:44)
	at io.grpc.internal.DelayedStream$4.run(DelayedStream.java:148)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:750)

Steps to reproduce the bug

Invoke remote gRPC services.

[jrhee17]

+1 We're also seeing the same issue:

GrpcServiceServerTest > reflectionService() FAILED
    java.lang.NoSuchMethodError: java.nio.ByteBuffer.clear()Ljava/nio/ByteBuffer;
        at io.grpc.internal.MessageFramer.writeKnownLengthUncompressed(MessageFramer.java:225)
        at io.grpc.internal.MessageFramer.writeUncompressed(MessageFramer.java:169)
        at io.grpc.internal.MessageFramer.writePayload(MessageFramer.java:142)
        at io.grpc.internal.AbstractStream.writeMessage(AbstractStream.java:66)
        at io.grpc.internal.ForwardingClientStream.writeMessage(ForwardingClientStream.java:37)
        at io.grpc.internal.DelayedStream$6.run(DelayedStream.java:283)
        at io.grpc.internal.DelayedStream.drainPendingCalls(DelayedStream.java:182)
        at io.grpc.internal.DelayedStream.access$100(DelayedStream.java:44)
        at io.grpc.internal.DelayedStream$4.run(DelayedStream.java:148)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:750)

ref: https://github.com/line/armeria/actions/runs/5706739708/job/15462723107?pr=5086#step:7:1636

[ejona86]

This was probably caused by #10299 .

[ejona86]

(I think there's some open issue in AnimalSniffer where it fails to catch stuff like this.)

[ejona86]

In order to verify:

./gradlew :grpc-core:build
unzip core/build/libs/grpc-core-1.58.0-SNAPSHOT.jar io/grpc/internal/MessageFramer.class
javap -c -p io/grpc/internal/MessageFramer.class | grep 'java/nio/ByteBuffer.clear:'

We want to see java/nio/ByteBuffer.clear:()Ljava/nio/Buffer; (specifically the return type Buffer, not ByteBuffer).

[cpovirk]

(I think there's some open issue in AnimalSniffer where it fails to catch stuff like this.)

#7360

(One day, I will have a conversation with you that is not prompted by a bug in a build tool :))

[ejona86]

I just sent out a fix. Not too scary. I verified that the ByteBuf.clear reference was correct in Java 8 and 11. And I also verified that io.grpc.Context remained using Java 7 bytecode. I also checked that Java 17 supports --release 7, but didn't actually do a build with it. That's important because in ~a month Java 11 no longer has active support and our build will probably need to move to Java 17.

@cpovirk, maybe some day, but likely not before Java 8 dies and JSpecify lands! (In related news, we need to fight the Gradle Module Metadata stuff with Guava 32.0.1! Don't be concerned yet; we still need to figure out which way is up.)