quarkus-infinispan-client Unable to read (unMarshall) a cached item before you write (Marshall) one.

[sean-scott-lr]

I have set up a remote Infinispan server. And have code that will read and write from a cache on that server.

The issue is that Quarkus is unable to unmarshall items from the cache, unless it has first marshaled into the cache.

Expected behavior
I am able to read cached items without first writing to the cache

Actual behavior
If you read a cached item before writing one, you get this exception

	at org.infinispan.client.hotrod.marshall.MarshallerUtil.bytes2obj(MarshallerUtil.java:47)
	at org.infinispan.client.hotrod.DataFormat.keyToObj(DataFormat.java:103)
	at org.infinispan.client.hotrod.impl.protocol.Codec21.readCacheEvent(Codec21.java:76)
	at org.infinispan.client.hotrod.impl.transport.netty.HeaderDecoder.decode(HeaderDecoder.java:153)
	at org.infinispan.client.hotrod.impl.transport.netty.HintedReplayingDecoder.callDecode(HintedReplayingDecoder.java:94)
	at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:278)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:359)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:345)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:337)
	at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:359)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:345)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:337)
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1408)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:359)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:345)
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:930)
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163)
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:677)
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:577)
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:531)
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:491)
	at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:905)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: protostream.com.google.protobuf.InvalidProtocolBufferException: Protocol message contained an invalid tag (zero).
	at protostream.com.google.protobuf.InvalidProtocolBufferException.invalidTag(InvalidProtocolBufferException.java:101)
	at protostream.com.google.protobuf.CodedInputStream.readTag(CodedInputStream.java:148)
	at org.infinispan.protostream.impl.RawProtoStreamReaderImpl.readTag(RawProtoStreamReaderImpl.java:45)
	at org.infinispan.protostream.WrappedMessage.readMessage(WrappedMessage.java:268)
	at org.infinispan.protostream.ProtobufUtil.fromWrappedByteArray(ProtobufUtil.java:165)
	at org.infinispan.query.remote.client.BaseProtoStreamMarshaller.objectFromByteBuffer(BaseProtoStreamMarshaller.java:33)
	at org.infinispan.commons.marshall.AbstractMarshaller.objectFromByteBuffer(AbstractMarshaller.java:82)
	at org.infinispan.client.hotrod.marshall.MarshallerUtil.bytes2obj(MarshallerUtil.java:31)
	... 25 more

To Reproduce
Steps to reproduce the behavior:

1. Follow the steps in the Using Infinispan Client guide. https://quarkus.io/guides/infinispan-client-guide
2. Create an Infinispan server with a cache in it
3. Start Quarkus and write a cache item to the cache
4. Restart Quarkus and read the item you write in step 3. Fail!

I believe that the Marshaller Beans are not loaded appropriately until you WRITE to a cache. Writing to the cache creates the bean... and if you don't do that first, then you can't unmarshall.

[avboy72]

I should note, that near cache must be enabled to reproduce the error

[avboy72]

It seems that the near cache is proactively loading upon startup, and encountering this error.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you!
We are doing this automatically to ensure out-of-date issues does not stay around indefinitely.
If you believe this issue is still relevant please put a comment on it on why and if it truly needs to stay request or add 'pinned' label.

[gsmet]

@wburns @karesti can you check this, please?

[karesti]

I could reproduce the error, I opened a JIRA in infinispan https://issues.redhat.com/browse/ISPN-11281

[sean-scott-lr]

Is no-one actually using the Infinispan extension, or is there some workaround I am unaware of?

[wburns]

I got a chance to look at this. The error is ignorable and has nothing to do with the get/put. It is the automatic registration of the protostream definitions.

The problem is that when registering the schemas for a second time it is already present and the server sends back a client event to tell the client that it has been modified, but can't deserialize this object (as it doesn't support this in near cache).

You can workaround this until a fix is done by adding the specific caches to the regex for near cache in the hotrod.client-properties file of infinispan.client.hotrod.near_cache.name_pattern=<regex> https://docs.jboss.org/infinispan/11.0/apidocs/org/infinispan/client/hotrod/configuration/package-summary.html#package.description This will prevent these other caches from being used with the near cache.

[sean-scott-lr]

I should have updated this after I determined the cause.

1. hotrod-client.properties should be in /resources/META-INF ( that wasn't documented in the guide )
2. the content of the file infinispan.client.hotrod.near_cache.name_pattern=^(?!___protobuf_metadata).*$

[wburns]

I should have updated this after I determined the cause.

1. hotrod-client.properties should be in /resources/META-INF ( that wasn't documented in the guide )
2. the content of the file infinispan.client.hotrod.near_cache.name_pattern=^(?!___protobuf_metadata).*$

👍