InvalidWireTypeException: Protocol message tag had invalid wire type.` #19529

yannick · 2023-02-15T11:15:31Z

yannick
Feb 15, 2023

we're seing
Message: org.apache.bookkeeper.mledger.ManagedLedgerException: com.google.protobuf.InvalidProtocolBufferException$InvalidWireTypeException: Protocol message tag had invalid wire type.
popping up on some partitions.
the partition is then inaccessible and the responsible broker sends a WARN org.apache.pulsar.broker.service.BrokerService - Failed to create topic

we're on 2.11.

i have no idea how to fix this.
can a partition be removed from a partitioned topic?

when i try to delete the topic:

pulsarctl topics delete persistent://tenant/events/mytopic

[✖]  code: 504 reason: 504 Gateway Timeout

michaeljmarshall · 2023-02-15T21:37:33Z

michaeljmarshall
Feb 15, 2023
Collaborator

@yannick - do you have a full stack trace? That will help determine where you're seeing the exception.

0 replies

yannick · 2023-02-16T04:39:09Z

yannick
Feb 16, 2023
Author

below a trace when i ran pulsar-admin topics stats-internal . on the broker side nothing is visible afair.


Message: org.apache.bookkeeper.mledger.ManagedLedgerException: com.google.protobuf.InvalidProtocolBufferException$InvalidWireTypeException: Protocol message tag had invalid wire type.

Stacktrace:

org.apache.pulsar.broker.service.BrokerServiceException$PersistenceException: org.apache.bookkeeper.mledger.ManagedLedgerException: com.google.protobuf.InvalidProtocolBufferException$InvalidWireTypeException: Protocol message tag had invalid wire type.
	at org.apache.pulsar.broker.service.BrokerService$2.openLedgerFailed(BrokerService.java:1483)
	at org.apache.bookkeeper.mledger.impl.ManagedLedgerFactoryImpl.lambda$asyncOpen$8(ManagedLedgerFactoryImpl.java:416)
	at java.base/java.util.concurrent.CompletableFuture.uniExceptionally(CompletableFuture.java:990)
	at java.base/java.util.concurrent.CompletableFuture$UniExceptionally.tryFire(CompletableFuture.java:974)
	at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:510)
	at java.base/java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2162)
	at org.apache.bookkeeper.mledger.impl.ManagedLedgerFactoryImpl$2.initializeFailed(ManagedLedgerFactoryImpl.java:411)
	at org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl$3$1.operationFailed(ManagedLedgerImpl.java:573)
	at org.apache.bookkeeper.mledger.impl.ManagedCursorImpl.lambda$recoverFromLedger$4(ManagedCursorImpl.java:521)
	at org.apache.bookkeeper.client.LedgerHandle$6.onSuccess(LedgerHandle.java:809)
	at org.apache.bookkeeper.client.LedgerHandle$6.onSuccess(LedgerHandle.java:806)
	at org.apache.bookkeeper.common.concurrent.FutureEventListener.accept(FutureEventListener.java:42)
	at org.apache.bookkeeper.common.concurrent.FutureEventListener.accept(FutureEventListener.java:26)
	at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863)
	at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841)
	at java.base/java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:482)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
	at java.base/java.lang.Thread.run(Thread.java:833)
Caused by: org.apache.bookkeeper.mledger.ManagedLedgerException: com.google.protobuf.InvalidProtocolBufferException$InvalidWireTypeException: Protocol message tag had invalid wire type.
Caused by: com.google.protobuf.InvalidProtocolBufferException$InvalidWireTypeException: Protocol message tag had invalid wire type.
	at com.google.protobuf.InvalidProtocolBufferException.invalidWireType(InvalidProtocolBufferException.java:134)
	at com.google.protobuf.UnknownFieldSet$Builder.mergeFieldFrom(UnknownFieldSet.java:527)
	at com.google.protobuf.GeneratedMessageV3$Builder.parseUnknownField(GeneratedMessageV3.java:863)
	at org.apache.bookkeeper.mledger.proto.MLDataFormats$PositionInfo$Builder.mergeFrom(MLDataFormats.java:8205)
	at org.apache.bookkeeper.mledger.proto.MLDataFormats$PositionInfo$1.parsePartialFrom(MLDataFormats.java:9215)
	at org.apache.bookkeeper.mledger.proto.MLDataFormats$PositionInfo$1.parsePartialFrom(MLDataFormats.java:9207)
	at com.google.protobuf.AbstractParser.parsePartialFrom(AbstractParser.java:158)
	at com.google.protobuf.AbstractParser.parseFrom(AbstractParser.java:191)
	at com.google.protobuf.AbstractParser.parseFrom(AbstractParser.java:203)
	at com.google.protobuf.AbstractParser.parseFrom(AbstractParser.java:208)
	at com.google.protobuf.AbstractParser.parseFrom(AbstractParser.java:48)
	at org.apache.bookkeeper.mledger.proto.MLDataFormats$PositionInfo.parseFrom(MLDataFormats.java:7788)
	at org.apache.bookkeeper.mledger.impl.ManagedCursorImpl.lambda$recoverFromLedger$4(ManagedCursorImpl.java:519)
	at org.apache.bookkeeper.client.LedgerHandle$6.onSuccess(LedgerHandle.java:809)
	at org.apache.bookkeeper.client.LedgerHandle$6.onSuccess(LedgerHandle.java:806)
	at org.apache.bookkeeper.common.concurrent.FutureEventListener.accept(FutureEventListener.java:42)
	at org.apache.bookkeeper.common.concurrent.FutureEventListener.accept(FutureEventListener.java:26)
	at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863)
	at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841)
	at java.base/java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:482)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
	at java.base/java.lang.Thread.run(Thread.java:833)


Reason:
 --- An unexpected error occurred in the server ---

Message: org.apache.bookkeeper.mledger.ManagedLedgerException: com.google.protobuf.InvalidProtocolBufferException$InvalidWireTypeException: Protocol message tag had invalid wire type.

Stacktrace:

org.apache.pulsar.broker.service.BrokerServiceException$PersistenceException: org.apache.bookkeeper.mledger.ManagedLedgerException: com.google.protobuf.InvalidProtocolBufferException$InvalidWireTypeException: Protocol message tag had invalid wire type.
	at org.apache.pulsar.broker.service.BrokerService$2.openLedgerFailed(BrokerService.java:1483)
	at org.apache.bookkeeper.mledger.impl.ManagedLedgerFactoryImpl.lambda$asyncOpen$8(ManagedLedgerFactoryImpl.java:416)
	at java.base/java.util.concurrent.CompletableFuture.uniExceptionally(CompletableFuture.java:990)
	at java.base/java.util.concurrent.CompletableFuture$UniExceptionally.tryFire(CompletableFuture.java:974)
	at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:510)
	at java.base/java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2162)
	at org.apache.bookkeeper.mledger.impl.ManagedLedgerFactoryImpl$2.initializeFailed(ManagedLedgerFactoryImpl.java:411)
	at org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl$3$1.operationFailed(ManagedLedgerImpl.java:573)
	at org.apache.bookkeeper.mledger.impl.ManagedCursorImpl.lambda$recoverFromLedger$4(ManagedCursorImpl.java:521)
	at org.apache.bookkeeper.client.LedgerHandle$6.onSuccess(LedgerHandle.java:809)
	at org.apache.bookkeeper.client.LedgerHandle$6.onSuccess(LedgerHandle.java:806)
	at org.apache.bookkeeper.common.concurrent.FutureEventListener.accept(FutureEventListener.java:42)
	at org.apache.bookkeeper.common.concurrent.FutureEventListener.accept(FutureEventListener.java:26)
	at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863)
	at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841)
	at java.base/java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:482)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
	at java.base/java.lang.Thread.run(Thread.java:833)
Caused by: org.apache.bookkeeper.mledger.ManagedLedgerException: com.google.protobuf.InvalidProtocolBufferException$InvalidWireTypeException: Protocol message tag had invalid wire type.
Caused by: com.google.protobuf.InvalidProtocolBufferException$InvalidWireTypeException: Protocol message tag had invalid wire type.
	at com.google.protobuf.InvalidProtocolBufferException.invalidWireType(InvalidProtocolBufferException.java:134)
	at com.google.protobuf.UnknownFieldSet$Builder.mergeFieldFrom(UnknownFieldSet.java:527)
	at com.google.protobuf.GeneratedMessageV3$Builder.parseUnknownField(GeneratedMessageV3.java:863)
	at org.apache.bookkeeper.mledger.proto.MLDataFormats$PositionInfo$Builder.mergeFrom(MLDataFormats.java:8205)
	at org.apache.bookkeeper.mledger.proto.MLDataFormats$PositionInfo$1.parsePartialFrom(MLDataFormats.java:9215)
	at org.apache.bookkeeper.mledger.proto.MLDataFormats$PositionInfo$1.parsePartialFrom(MLDataFormats.java:9207)
	at com.google.protobuf.AbstractParser.parsePartialFrom(AbstractParser.java:158)
	at com.google.protobuf.AbstractParser.parseFrom(AbstractParser.java:191)
	at com.google.protobuf.AbstractParser.parseFrom(AbstractParser.java:203)
	at com.google.protobuf.AbstractParser.parseFrom(AbstractParser.java:208)
	at com.google.protobuf.AbstractParser.parseFrom(AbstractParser.java:48)
	at org.apache.bookkeeper.mledger.proto.MLDataFormats$PositionInfo.parseFrom(MLDataFormats.java:7788)
	at org.apache.bookkeeper.mledger.impl.ManagedCursorImpl.lambda$recoverFromLedger$4(ManagedCursorImpl.java:519)
	at org.apache.bookkeeper.client.LedgerHandle$6.onSuccess(LedgerHandle.java:809)
	at org.apache.bookkeeper.client.LedgerHandle$6.onSuccess(LedgerHandle.java:806)
	at org.apache.bookkeeper.common.concurrent.FutureEventListener.accept(FutureEventListener.java:42)
	at org.apache.bookkeeper.common.concurrent.FutureEventListener.accept(FutureEventListener.java:26)
	at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863)
	at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841)
	at java.base/java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:482)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
	at java.base/java.lang.Thread.run(Thread.java:833)

0 replies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

InvalidWireTypeException: Protocol message tag had invalid wire type.` #19529

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{editor}}'s edit

{{editor}}'s edit

Uh oh!

Replies: 2 comments

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Select a reply

Uh oh!

InvalidWireTypeException: Protocol message tag had invalid wire type.` #19529

Uh oh!

Uh oh!

yannick Feb 15, 2023

Replies: 2 comments

Uh oh!

michaeljmarshall Feb 15, 2023 Collaborator

Uh oh!

yannick Feb 16, 2023 Author

yannick
Feb 15, 2023

michaeljmarshall
Feb 15, 2023
Collaborator

yannick
Feb 16, 2023
Author